<!DOCTYPE HTML>
<html lang="zh-CN">


<head>
    <meta charset="utf-8">
    <meta name="keywords" content="爬虫实战：高清图片下载, 张亚飞,飞凡空间,国学,Python,Go">
    <meta name="description" content="Hello，小伙子们，小姑娘们，我又要出新教程了！本次教程为Python爬虫实战：高清图片下载，主要针对刚学完Python基础的同学，或者想学习爬虫方向的年轻人，很合适作为爬虫入们教程，难度虽然不高，但小项目仍然蕴含有大智慧。
项目演示演示">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=no">
    <meta name="renderer" content="webkit|ie-stand|ie-comp">
    <meta name="mobile-web-app-capable" content="yes">
    <meta name="format-detection" content="telephone=no">
    <meta name="apple-mobile-web-app-capable" content="yes">
    <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
	<meta name="baidu-site-verification" content="code-w2ezfaoXE0" />
    <title>爬虫实战：高清图片下载 | 飞凡空间</title>
    <link rel="icon" type="image/png" href="/medias/my-logo.png">

    <link rel="stylesheet" type="text/css" href="/libs/awesome/css/all.css">
    <link rel="stylesheet" type="text/css" href="/libs/materialize/materialize.min.css">
    <link rel="stylesheet" type="text/css" href="/libs/aos/aos.css">
    <link rel="stylesheet" type="text/css" href="/libs/animate/animate.min.css">
    <link rel="stylesheet" type="text/css" href="/libs/lightGallery/css/lightgallery.min.css">
    <link rel="stylesheet" type="text/css" href="/css/matery.css">
    <link rel="stylesheet" type="text/css" href="/css/my.css">
    
    <script src="/libs/jquery/jquery.min.js"></script>
    
<meta name="generator" content="Hexo 5.2.0">
<style>.github-emoji { position: relative; display: inline-block; width: 1.2em; min-height: 1.2em; overflow: hidden; vertical-align: top; color: transparent; }  .github-emoji > span { position: relative; z-index: 10; }  .github-emoji img, .github-emoji .fancybox { margin: 0 !important; padding: 0 !important; border: none !important; outline: none !important; text-decoration: none !important; user-select: none !important; cursor: auto !important; }  .github-emoji img { height: 1.2em !important; width: 1.2em !important; position: absolute !important; left: 50% !important; top: 50% !important; transform: translate(-50%, -50%) !important; user-select: none !important; cursor: auto !important; } .github-emoji-fallback { color: inherit; } .github-emoji-fallback img { opacity: 0 !important; }</style>
<link rel="alternate" href="/atom.xml" title="飞凡空间" type="application/atom+xml">
<link rel="stylesheet" href="/css/prism-tomorrow.css" type="text/css">
<link rel="stylesheet" href="/css/prism-line-numbers.css" type="text/css"></head>


<body>
    <header class="navbar-fixed">
    <nav id="headNav" class="bg-color nav-transparent">
        <div id="navContainer" class="nav-wrapper container">
            <div class="brand-logo">
                <a href="/" class="waves-effect waves-light">
                    
                    <img src="/medias/my-logo.png" class="logo-img" alt="LOGO">
                    
                    <span class="logo-span">飞凡空间</span>
                </a>
            </div>
            

<a href="#" data-target="mobile-nav" class="sidenav-trigger button-collapse"><i class="fas fa-bars"></i></a>
<ul class="right nav-menu">
  
  <li class="hide-on-med-and-down nav-item">
    
    <a href="/" class="waves-effect waves-light">
      
      <i class="fas fa-home" style="zoom: 0.6;"></i>
      
      <span>首页</span>
    </a>
    
  </li>
  
  <li class="hide-on-med-and-down nav-item">
    
    <a href="/tags" class="waves-effect waves-light">
      
      <i class="fas fa-tags" style="zoom: 0.6;"></i>
      
      <span>标签</span>
    </a>
    
  </li>
  
  <li class="hide-on-med-and-down nav-item">
    
    <a href="/categories" class="waves-effect waves-light">
      
      <i class="fas fa-bookmark" style="zoom: 0.6;"></i>
      
      <span>分类</span>
    </a>
    
  </li>
  
  <li class="hide-on-med-and-down nav-item">
    
    <a href="/archives" class="waves-effect waves-light">
      
      <i class="fas fa-archive" style="zoom: 0.6;"></i>
      
      <span>归档</span>
    </a>
    
  </li>
  
  <li class="hide-on-med-and-down nav-item">
    
    <a href="/about" class="waves-effect waves-light">
      
      <i class="fas fa-user-circle" style="zoom: 0.6;"></i>
      
      <span>关于</span>
    </a>
    
  </li>
  
  <li class="hide-on-med-and-down nav-item">
    
    <a href="/friends" class="waves-effect waves-light">
      
      <i class="fas fa-address-book" style="zoom: 0.6;"></i>
      
      <span>友情链接</span>
    </a>
    
  </li>
  
  <li class="hide-on-med-and-down nav-item">
    
    <a href="/contact" class="waves-effect waves-light">
      
      <i class="fas fa-comments" style="zoom: 0.6;"></i>
      
      <span>留言板</span>
    </a>
    
  </li>
  
  <li>
    <a href="#searchModal" class="modal-trigger waves-effect waves-light">
      <i id="searchIcon" class="fas fa-search" title="搜索" style="zoom: 0.85;"></i>
    </a>
  </li>
</ul>

<div id="mobile-nav" class="side-nav sidenav">

    <div class="mobile-head bg-color">
        
        <img src="/medias/my-logo.png" class="logo-img circle responsive-img">
        
        <div class="logo-name">飞凡空间</div>
        <div class="logo-desc">
            
            诚者，天之道也；诚之者，人之道也。诚者，不勉而中，不思而得。从容中道，圣人也；诚之者，择善而固执者也。
            
        </div>
    </div>

    

    <ul class="menu-list mobile-menu-list">
        
        <li class="m-nav-item">
	  
		<a href="/" class="waves-effect waves-light">
			
			    <i class="fa-fw fas fa-home"></i>
			
			首页
		</a>
          
        </li>
        
        <li class="m-nav-item">
	  
		<a href="/tags" class="waves-effect waves-light">
			
			    <i class="fa-fw fas fa-tags"></i>
			
			标签
		</a>
          
        </li>
        
        <li class="m-nav-item">
	  
		<a href="/categories" class="waves-effect waves-light">
			
			    <i class="fa-fw fas fa-bookmark"></i>
			
			分类
		</a>
          
        </li>
        
        <li class="m-nav-item">
	  
		<a href="/archives" class="waves-effect waves-light">
			
			    <i class="fa-fw fas fa-archive"></i>
			
			归档
		</a>
          
        </li>
        
        <li class="m-nav-item">
	  
		<a href="/about" class="waves-effect waves-light">
			
			    <i class="fa-fw fas fa-user-circle"></i>
			
			关于
		</a>
          
        </li>
        
        <li class="m-nav-item">
	  
		<a href="/friends" class="waves-effect waves-light">
			
			    <i class="fa-fw fas fa-address-book"></i>
			
			友情链接
		</a>
          
        </li>
        
        <li class="m-nav-item">
	  
		<a href="/contact" class="waves-effect waves-light">
			
			    <i class="fa-fw fas fa-comments"></i>
			
			留言板
		</a>
          
        </li>
        
        
    </ul>
</div>

        </div>

        
    </nav>

</header>

    



<div class="bg-cover pd-header post-cover" style="background-image: url('https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/Python/blog/advanced-6.jpg')">
    <div class="container" style="right: 0px;left: 0px;">
        <div class="row">
            <div class="col s12 m12 l12">
                <div class="brand">
                    <div class="description center-align post-title">
                        爬虫实战：高清图片下载
                    </div>
                </div>
            </div>
        </div>
    </div>
</div>




<main class="post-container content">

    
    <link rel="stylesheet" href="/libs/tocbot/tocbot.css">
<style>
    #articleContent h1::before,
    #articleContent h2::before,
    #articleContent h3::before,
    #articleContent h4::before,
    #articleContent h5::before,
    #articleContent h6::before {
        display: block;
        content: " ";
        height: 100px;
        margin-top: -100px;
        visibility: hidden;
    }

    #articleContent :focus {
        outline: none;
    }

    .toc-fixed {
        position: fixed;
        top: 64px;
    }

    .toc-widget {
        padding-left: 20px;
    }

    .toc-widget .toc-title {
        margin: 35px 0 15px 0;
        padding-left: 17px;
        font-size: 1.5rem;
        font-weight: bold;
        line-height: 1.5rem;
    }

    .toc-widget ol {
        padding: 0;
        list-style: none;
    }

    #toc-content ol {
        padding-left: 10px;
    }

    #toc-content ol li {
        padding-left: 10px;
    }

    #toc-content .toc-link:hover {
        color: #42b983;
        font-weight: 700;
        text-decoration: underline;
    }

    #toc-content .toc-link::before {
        background-color: transparent;
        max-height: 25px;
    }

    #toc-content .is-active-link {
        color: #42b983;
    }

    #toc-content .is-active-link::before {
        background-color: #42b983;
    }

    #floating-toc-btn {
        position: fixed;
        right: 15px;
        bottom: 76px;
        padding-top: 15px;
        margin-bottom: 0;
        z-index: 998;
    }

    #floating-toc-btn .btn-floating {
        width: 48px;
        height: 48px;
    }

    #floating-toc-btn .btn-floating i {
        line-height: 48px;
        font-size: 1.4rem;
    }
</style>
<div class="row">
    <div id="main-content" class="col s12 m12 l9">
        <!-- 文章内容详情 -->
<div id="artDetail">
    <div class="card">
        <div class="card-content article-info">
            <div class="row tag-cate">
                <div class="col s7">
                    
                    <div class="article-tag">
                        
                            <a href="/tags/Python%E4%B9%8B%E8%B7%AF/">
                                <span class="chip bg-color">Python之路</span>
                            </a>
                        
                            <a href="/tags/%E8%BF%9B%E9%98%B6%E7%AF%87/">
                                <span class="chip bg-color">进阶篇</span>
                            </a>
                        
                            <a href="/tags/%E7%BD%91%E7%BB%9C%E7%88%AC%E8%99%AB/">
                                <span class="chip bg-color">网络爬虫</span>
                            </a>
                        
                            <a href="/tags/%E9%AB%98%E6%B8%85%E5%9B%BE%E7%89%87%E4%B8%8B%E8%BD%BD/">
                                <span class="chip bg-color">高清图片下载</span>
                            </a>
                        
                    </div>
                    
                </div>
                <div class="col s5 right-align">
                    
                    <div class="post-cate">
                        <i class="fas fa-bookmark fa-fw icon-category"></i>
                        
                            <a href="/categories/Python/" class="post-category">
                                Python
                            </a>
                        
                            <a href="/categories/Python/Advanced/" class="post-category">
                                Advanced
                            </a>
                        
                    </div>
                    
                </div>
            </div>

            <div class="post-info">
                
                <div class="post-date info-break-policy">
                    <i class="far fa-calendar-minus fa-fw"></i>发布日期:&nbsp;&nbsp;
                    2021-04-06
                </div>
                

                

                
                <div class="info-break-policy">
                    <i class="far fa-file-word fa-fw"></i>文章字数:&nbsp;&nbsp;
                    8.4k
                </div>
                

                
                <div class="info-break-policy">
                    <i class="far fa-clock fa-fw"></i>阅读时长:&nbsp;&nbsp;
                    40 分
                </div>
                
				
                
                    <div id="busuanzi_container_page_pv" class="info-break-policy">
                        <i class="far fa-eye fa-fw"></i>阅读次数:&nbsp;&nbsp;
                        <span id="busuanzi_value_page_pv"></span>
                    </div>
				
            </div>
            
        </div>
        <hr class="clearfix">
        <div class="card-content article-card-content">
            <div id="articleContent">
                <p>Hello，小伙子们，小姑娘们，我又要出新教程了！本次教程为Python爬虫实战：高清图片下载，主要针对刚学完Python基础的同学，或者想学习爬虫方向的年轻人，很合适作为爬虫入们教程，难度虽然不高，但小项目仍然蕴含有大智慧。</p>
<h2 id="项目演示"><a href="#项目演示" class="headerlink" title="项目演示"></a>项目演示</h2><p>演示视频：<a target="_blank" rel="noopener" href="https://www.bilibili.com/video/BV14i4y1N7pP/">https://www.bilibili.com/video/BV14i4y1N7pP/</a></p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/spider_object.gif"></p>
<p>项目代码：<a target="_blank" rel="noopener" href="https://gitee.com/zhangyafeii/python-doc-tutorial/tree/master/code/%E7%BD%91%E7%BB%9C%E7%88%AC%E8%99%AB/%E7%88%AC%E8%99%AB%E5%AE%9E%E6%88%98%EF%BC%9A%E9%AB%98%E6%B8%85%E5%9B%BE%E7%89%87%E4%B8%8B%E8%BD%BD">爬虫实战：高清图片下载</a></p>
<p><strong>本教程将用到的知识点有</strong>：</p>
<ul>
<li><strong><code>requests</code>的使用</strong></li>
<li><strong><code>lxml</code>库解析<code>html</code></strong></li>
<li><strong><code>json</code>数据解析</strong></li>
<li><strong>图片数据存储</strong></li>
<li><strong>请求过滤</strong></li>
<li><strong>进度条打印</strong></li>
<li><strong>多线程提速</strong></li>
<li><strong>其他</strong></li>
</ul>
<h2 id="项目一：NET牛人"><a href="#项目一：NET牛人" class="headerlink" title="项目一：NET牛人"></a>项目一：NET牛人</h2><h3 id="一、目标网站分析"><a href="#一、目标网站分析" class="headerlink" title="一、目标网站分析"></a>一、目标网站分析</h3><p>网址：<a target="_blank" rel="noopener" href="https://ss.netnr.com/wallpaper">https://ss.netnr.com/wallpaper</a></p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210405214028818.png"></p>
<p>该网站提供了各种格式的高清壁纸，用户可以根据自己的需要选择相应格式下载，鼠标向下拉会自动刷新获取更多壁纸，该网站有4K专区、美女模特、风景大片等18种壁纸分类，默认为最新壁纸。</p>
<p>我们的目标就是：使用Python自动化下载该网站的高清壁纸，可以指定类别和下载数量。当然，有兴趣的伙伴也可以指定壁纸分辨率，本教程将按照该网站所提供壁纸的最高分辨率进行下载。</p>
<h3 id="二、网络请求分析"><a href="#二、网络请求分析" class="headerlink" title="二、网络请求分析"></a>二、网络请求分析</h3><p>右键点击 - “检查 - 网络”，刷新页面，观察请求变化，结果在XHR-ajax请求中发现了每次获取新数据都会发送的请求，</p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210405221726608.png"></p>
<p>我们再观察一下它的返回信息，果然就是我们需要的图片信息。返回数据为json格式，data中包含了12个元素，每个元素都是一张图片的信息，其中包含可以获取的所有分辨率的图片地址，默认url为最高分辨率，我们观察到，还有两个重要的元素，id和tag, id是唯一标示，tag是图片的描述，这两个值我们在给图片命名的时候会用到。</p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210405222247528.png"></p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210405222351234.png"></p>
<p>我们再来观察一下请求信息，发现请求url为<code>https://bird.ioliu.cn/v2</code>，GET请求，每会携带一个参数url，<code>http://wallpaper.apc.360.cn/index.php?c=WallPaper&amp;start=13&amp;count=12&amp;from=360chrome&amp;a=getAppsByOrder&amp;order=create_time</code>,其中参数start和count分别是起始位置和数量的意思，另外，如果没有选择分类，默认是按时间排序，如果选择分类，<code>http://wallpaper.apc.360.cn/index.php?c=WallPaper&amp;start=1&amp;count=12&amp;from=360chrome&amp;a=getAppsByCategory&amp;cid=36</code>，将会有一个参数<code>cid</code>，值为分类<code>id</code>。我们在页面右上角可以选择壁纸分类，当我们点击分类的时候，页面url对应的<code>#id</code>都会发生变化。</p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210406110309727.png"></p>
<p>总结：有三个重要的参数，start：起始位置，count：请求数量，<code>cid</code>：分类ID, 若要获取最新壁纸，order=create_time</p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210405222838077.png"></p>
<p>经过上述网络请求分析，可以看出本次任务请求构造和逻辑较简单，那么接下来我们将去实现它。</p>
<h3 id="三、初步实现"><a href="#三、初步实现" class="headerlink" title="三、初步实现"></a>三、初步实现</h3><ul>
<li><p>代码</p>
<pre class="line-numbers language-python"><code class="language-python"><span class="token keyword">import</span> requests

url <span class="token operator">=</span> <span class="token string">'https://bird.ioliu.cn/v2'</span>

start <span class="token operator">=</span> <span class="token number">1</span>
count <span class="token operator">=</span> <span class="token number">12</span>

params <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token string">'url'</span><span class="token punctuation">:</span> f<span class="token string">'http://wallpaper.apc.360.cn/index.php?c=WallPaper&amp;start={start}&amp;count={count}&amp;from=360chrome&amp;a=getAppsByOrder&amp;order=create_time'</span><span class="token punctuation">}</span>

response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>url<span class="token operator">=</span>url<span class="token punctuation">,</span> params<span class="token operator">=</span>params<span class="token punctuation">)</span>
result <span class="token operator">=</span> response<span class="token punctuation">.</span>json<span class="token punctuation">(</span><span class="token punctuation">)</span>

<span class="token keyword">for</span> image_info <span class="token keyword">in</span> result<span class="token punctuation">[</span><span class="token string">'data'</span><span class="token punctuation">]</span><span class="token punctuation">:</span>
    uid <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'id'</span><span class="token punctuation">]</span>
    image_url <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'url'</span><span class="token punctuation">]</span>
    tags <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'tag'</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'_category_'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'_全部_'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span>strip<span class="token punctuation">(</span><span class="token string">'_ '</span><span class="token punctuation">)</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">' '</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
    size <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'resolution'</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'x'</span><span class="token punctuation">,</span> <span class="token string">'_'</span><span class="token punctuation">)</span>
    <span class="token keyword">print</span><span class="token punctuation">(</span>image_url<span class="token punctuation">,</span> tags<span class="token punctuation">,</span> size<span class="token punctuation">,</span> uid<span class="token punctuation">)</span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
</li>
<li><p>运行结果</p>
<pre class="line-numbers language-shell"><code class="language-shell">λ python spider.py
http://browser9.qhimg.com/bdr/__85/t01753453b660de14e9.jpg 汽车天下 1920_1080 2008041
http://browser9.qhimg.com/bdr/__85/t01bbd94b90e850d1d3.jpg 动感水果_车厘子_小清新 1920_1080 2008033
http://browser9.qhimg.com/bdr/__85/t019fd908f724f51900.jpg 海洋天堂_沙滩_风景大片 1920_1080 2008032
http://browser9.qhimg.com/bdr/__85/t010448c46c1ecf7cab.jpg 卡通人物_动漫卡通 1920_1200 2007786
http://browser9.qhimg.com/bdr/__85/t0183def7a3a7924215.jpg 自然风光_瀑布_风景大片 1920_1080 2007276
http://browser9.qhimg.com/bdr/__85/t016ad88ddaf2ae2d92.jpg 雪山_天空__风景大片 2560_1440 2007035
http://browser9.qhimg.com/bdr/__85/t0179b947962a684673.jpg 娇艳欲滴_茶花_小清新 1920_1080 2007033
http://browser9.qhimg.com/bdr/__85/t01cd97ec806b712059.jpg 未来城市_航天飞机_4K专区 3840_2160 2007028
http://browser9.qhimg.com/bdr/__85/t018160b069da5cac0d.jpg 落日余晖_花海_风景大片 1920_1080 2007013
http://browser9.qhimg.com/bdr/__85/t013a4ed4683039d101.jpg 自行车_街道_动漫卡通 1920_1080 2007011
http://browser9.qhimg.com/bdr/__85/t01a78941bc25ae2cf9.jpg 兔子_萌宠动物 1920_1080 2005698
http://browser9.qhimg.com/bdr/__85/t01b85e62ab512342e5.jpg 落日余晖_秧田_风景大片 1920_1080 2005696<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
<p>按照我们之前的分析进行代码编写，运行之后发现整个流程没有任何问题，我们成功获取到了图片的id、url、tags和size。核心步骤实现了之后，我们就要开始完善细节部分了。</p>
</li>
</ul>
<h3 id="四、自定义参数和图片存储"><a href="#四、自定义参数和图片存储" class="headerlink" title="四、自定义参数和图片存储"></a>四、自定义参数和图片存储</h3><ul>
<li><p>代码</p>
<pre class="line-numbers language-python"><code class="language-python"><span class="token keyword">import</span> os

<span class="token keyword">import</span> requests

image_type_info <span class="token operator">=</span> <span class="token punctuation">{</span>
        <span class="token number">0</span><span class="token punctuation">:</span> <span class="token string">'最新壁纸'</span><span class="token punctuation">,</span> <span class="token number">6</span><span class="token punctuation">:</span> <span class="token string">'美女模特'</span><span class="token punctuation">,</span> <span class="token number">30</span><span class="token punctuation">:</span> <span class="token string">'爱情美图'</span><span class="token punctuation">,</span> <span class="token number">9</span><span class="token punctuation">:</span> <span class="token string">'风景大片'</span><span class="token punctuation">,</span>
        <span class="token number">36</span><span class="token punctuation">:</span> <span class="token string">'4K专区'</span><span class="token punctuation">,</span> <span class="token number">15</span><span class="token punctuation">:</span> <span class="token string">'小清新'</span><span class="token punctuation">,</span> <span class="token number">26</span><span class="token punctuation">:</span> <span class="token string">'动漫卡通'</span><span class="token punctuation">,</span> <span class="token number">11</span><span class="token punctuation">:</span> <span class="token string">'明星风尚'</span><span class="token punctuation">,</span>
        <span class="token number">14</span><span class="token punctuation">:</span> <span class="token string">'萌宠动物'</span><span class="token punctuation">,</span> <span class="token number">5</span><span class="token punctuation">:</span> <span class="token string">'游戏壁纸'</span><span class="token punctuation">,</span> <span class="token number">12</span><span class="token punctuation">:</span> <span class="token string">'汽车天下'</span><span class="token punctuation">,</span> <span class="token number">10</span><span class="token punctuation">:</span> <span class="token string">'炫酷时尚'</span><span class="token punctuation">,</span>
        <span class="token number">29</span><span class="token punctuation">:</span> <span class="token string">'月历壁纸'</span><span class="token punctuation">,</span> <span class="token number">7</span><span class="token punctuation">:</span> <span class="token string">'景视剧照'</span><span class="token punctuation">,</span> <span class="token number">35</span><span class="token punctuation">:</span> <span class="token string">'文字控'</span><span class="token punctuation">,</span> <span class="token number">13</span><span class="token punctuation">:</span> <span class="token string">'节日美图'</span><span class="token punctuation">,</span>
        <span class="token number">22</span><span class="token punctuation">:</span> <span class="token string">'军事天地'</span><span class="token punctuation">,</span> <span class="token number">16</span><span class="token punctuation">:</span> <span class="token string">'劲爆体育'</span><span class="token punctuation">,</span> <span class="token number">18</span><span class="token punctuation">:</span> <span class="token string">'BABY秀'</span>
<span class="token punctuation">}</span>

<span class="token keyword">def</span> <span class="token function">main</span><span class="token punctuation">(</span>start<span class="token punctuation">:</span> int<span class="token punctuation">,</span> count<span class="token punctuation">:</span> int<span class="token punctuation">,</span> image_type<span class="token punctuation">:</span> int<span class="token punctuation">)</span><span class="token punctuation">:</span>
    url <span class="token operator">=</span> <span class="token string">'https://bird.ioliu.cn/v2'</span>

    <span class="token keyword">if</span> image_type <span class="token operator">==</span> <span class="token number">0</span><span class="token punctuation">:</span>
        params <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token string">'url'</span><span class="token punctuation">:</span> f<span class="token string">'http://wallpaper.apc.360.cn/index.php?c=WallPaper&amp;start={start}&amp;count={count}&amp;from=360chrome&amp;a=getAppsByOrder&amp;order=create_time'</span><span class="token punctuation">}</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        params <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token string">'url'</span><span class="token punctuation">:</span> f<span class="token string">'http://wallpaper.apc.360.cn/index.php?c=WallPaper&amp;start={start}&amp;count={count}&amp;from=360chrome&amp;a=getAppsByCategory&amp;cid={image_type}'</span><span class="token punctuation">}</span>

    <span class="token comment" spellcheck="true"># 图片文件夹分类</span>
    image_type_desc <span class="token operator">=</span> image_type_info<span class="token punctuation">[</span>image_type<span class="token punctuation">]</span>
    <span class="token keyword">if</span> <span class="token operator">not</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>exists<span class="token punctuation">(</span>f<span class="token string">'images/{image_type_desc}'</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        os<span class="token punctuation">.</span>makedirs<span class="token punctuation">(</span>f<span class="token string">'images/{image_type_desc}'</span><span class="token punctuation">)</span>

    <span class="token comment" spellcheck="true"># 发送请求</span>
    response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>url<span class="token operator">=</span>url<span class="token punctuation">,</span> params<span class="token operator">=</span>params<span class="token punctuation">)</span>
    result <span class="token operator">=</span> response<span class="token punctuation">.</span>json<span class="token punctuation">(</span><span class="token punctuation">)</span>

    <span class="token keyword">for</span> image_info <span class="token keyword">in</span> result<span class="token punctuation">[</span><span class="token string">'data'</span><span class="token punctuation">]</span><span class="token punctuation">:</span>
        uid <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'id'</span><span class="token punctuation">]</span>
        image_url <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'url'</span><span class="token punctuation">]</span>
        tags <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'tag'</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'_category_'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'_全部_'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span>strip<span class="token punctuation">(</span><span class="token string">'_ '</span><span class="token punctuation">)</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">' '</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
        size <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'resolution'</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'x'</span><span class="token punctuation">,</span> <span class="token string">'_'</span><span class="token punctuation">)</span>
        filepath <span class="token operator">=</span> f<span class="token string">'images/{image_type_info[image_type]}/{tags}_{uid}_{size}.jpg'</span>
        response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>image_url<span class="token punctuation">)</span>
        <span class="token keyword">with</span> open<span class="token punctuation">(</span>filepath<span class="token punctuation">,</span> mode<span class="token operator">=</span><span class="token string">'wb'</span><span class="token punctuation">)</span> <span class="token keyword">as</span> f<span class="token punctuation">:</span>
            f<span class="token punctuation">.</span>write<span class="token punctuation">(</span>response<span class="token punctuation">.</span>content<span class="token punctuation">)</span>
        <span class="token keyword">print</span><span class="token punctuation">(</span>f<span class="token string">'{image_url} {filepath} 下载成功'</span><span class="token punctuation">)</span>

<span class="token keyword">if</span> __name__ <span class="token operator">==</span> <span class="token string">'__main__'</span><span class="token punctuation">:</span>
    main<span class="token punctuation">(</span>start<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">,</span> count<span class="token operator">=</span><span class="token number">12</span><span class="token punctuation">,</span> image_type<span class="token operator">=</span><span class="token number">0</span><span class="token punctuation">)</span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
</li>
<li><p>运行结果</p>
<pre class="line-numbers language-shell"><code class="language-shell">λ python spider.py
http://browser9.qhimg.com/bdr/__85/t01753453b660de14e9.jpg images/最新壁纸/汽车天下_2008041_1920_1080.jpg 下载成功
http://browser9.qhimg.com/bdr/__85/t01bbd94b90e850d1d3.jpg images/最新壁纸/动感水果_车厘子_小清新_2008033_1920_1080.jpg 下载成功
http://browser9.qhimg.com/bdr/__85/t019fd908f724f51900.jpg images/最新壁纸/海洋天堂_沙滩_风景大片_2008032_1920_1080.jpg 下载成功
http://browser9.qhimg.com/bdr/__85/t010448c46c1ecf7cab.jpg images/最新壁纸/卡通人物_动漫卡通_2007786_1920_1200.jpg 下载成功
http://browser9.qhimg.com/bdr/__85/t0183def7a3a7924215.jpg images/最新壁纸/自然风光_瀑布_风景大片_2007276_1920_1080.jpg 下载成功
http://browser9.qhimg.com/bdr/__85/t016ad88ddaf2ae2d92.jpg images/最新壁纸/雪山_天空__风景大片_2007035_2560_1440.jpg 下载成功
http://browser9.qhimg.com/bdr/__85/t0179b947962a684673.jpg images/最新壁纸/娇艳欲滴_茶花_小清新_2007033_1920_1080.jpg 下载成功
http://browser9.qhimg.com/bdr/__85/t01cd97ec806b712059.jpg images/最新壁纸/未来城市_航天飞机_4K专区_2007028_3840_2160.jpg 下载成功
http://browser9.qhimg.com/bdr/__85/t018160b069da5cac0d.jpg images/最新壁纸/落日余晖_花海_风景大片_2007013_1920_1080.jpg 下载成功
http://browser9.qhimg.com/bdr/__85/t013a4ed4683039d101.jpg images/最新壁纸/自行车_街道_动漫卡通_2007011_1920_1080.jpg 下载成功
http://browser9.qhimg.com/bdr/__85/t01a78941bc25ae2cf9.jpg images/最新壁纸/兔子_萌宠动物_2005698_1920_1080.jpg 下载成功
http://browser9.qhimg.com/bdr/__85/t01b85e62ab512342e5.jpg images/最新壁纸/落日余晖_秧田_风景大片_2005696_1920_1080.jpg 下载成功<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
<p>当前目录下会生成images和最新壁纸两个文件夹，里面有刚下载的12张图片。</p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210406111452340.png"></p>
</li>
</ul>
<p>至此，我们已经实现了项目的核心下载功能，但，还不够完善。</p>
<h3 id="五、功能完善"><a href="#五、功能完善" class="headerlink" title="五、功能完善"></a>五、功能完善</h3><h4 id="1-用户提示信息"><a href="#1-用户提示信息" class="headerlink" title="1. 用户提示信息"></a>1. 用户提示信息</h4><pre class="line-numbers language-python"><code class="language-python"><span class="token keyword">from</span> prettytable <span class="token keyword">import</span> PrettyTable

image_type_info <span class="token operator">=</span> <span class="token punctuation">{</span>
    <span class="token number">0</span><span class="token punctuation">:</span> <span class="token string">'最新壁纸'</span><span class="token punctuation">,</span> <span class="token number">6</span><span class="token punctuation">:</span> <span class="token string">'美女模特'</span><span class="token punctuation">,</span> <span class="token number">30</span><span class="token punctuation">:</span> <span class="token string">'爱情美图'</span><span class="token punctuation">,</span> <span class="token number">9</span><span class="token punctuation">:</span> <span class="token string">'风景大片'</span><span class="token punctuation">,</span>
    <span class="token number">36</span><span class="token punctuation">:</span> <span class="token string">'4K专区'</span><span class="token punctuation">,</span> <span class="token number">15</span><span class="token punctuation">:</span> <span class="token string">'小清新'</span><span class="token punctuation">,</span> <span class="token number">26</span><span class="token punctuation">:</span> <span class="token string">'动漫卡通'</span><span class="token punctuation">,</span> <span class="token number">11</span><span class="token punctuation">:</span> <span class="token string">'明星风尚'</span><span class="token punctuation">,</span>
    <span class="token number">14</span><span class="token punctuation">:</span> <span class="token string">'萌宠动物'</span><span class="token punctuation">,</span> <span class="token number">5</span><span class="token punctuation">:</span> <span class="token string">'游戏壁纸'</span><span class="token punctuation">,</span> <span class="token number">12</span><span class="token punctuation">:</span> <span class="token string">'汽车天下'</span><span class="token punctuation">,</span> <span class="token number">10</span><span class="token punctuation">:</span> <span class="token string">'炫酷时尚'</span><span class="token punctuation">,</span>
    <span class="token number">29</span><span class="token punctuation">:</span> <span class="token string">'月历壁纸'</span><span class="token punctuation">,</span> <span class="token number">7</span><span class="token punctuation">:</span> <span class="token string">'景视剧照'</span><span class="token punctuation">,</span> <span class="token number">35</span><span class="token punctuation">:</span> <span class="token string">'文字控'</span><span class="token punctuation">,</span> <span class="token number">13</span><span class="token punctuation">:</span> <span class="token string">'节日美图'</span><span class="token punctuation">,</span>
    <span class="token number">22</span><span class="token punctuation">:</span> <span class="token string">'军事天地'</span><span class="token punctuation">,</span> <span class="token number">16</span><span class="token punctuation">:</span> <span class="token string">'劲爆体育'</span><span class="token punctuation">,</span> <span class="token number">18</span><span class="token punctuation">:</span> <span class="token string">'BABY秀'</span>
<span class="token punctuation">}</span>

<span class="token keyword">def</span> <span class="token function">init_tips_info</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    table <span class="token operator">=</span> PrettyTable<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token string">'序号1'</span><span class="token punctuation">,</span> <span class="token string">'壁纸分类1'</span><span class="token punctuation">,</span> <span class="token string">'序号2'</span><span class="token punctuation">,</span> <span class="token string">'壁纸分类2'</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
    row <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span>
    <span class="token keyword">for</span> index<span class="token punctuation">,</span> key <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>image_type_info<span class="token punctuation">,</span> start<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        row<span class="token punctuation">.</span>extend<span class="token punctuation">(</span><span class="token punctuation">[</span>key<span class="token punctuation">,</span> image_type_info<span class="token punctuation">[</span>key<span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
        <span class="token keyword">if</span> len<span class="token punctuation">(</span>row<span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token number">4</span><span class="token punctuation">:</span>
            table<span class="token punctuation">.</span>add_row<span class="token punctuation">(</span>row<span class="token punctuation">)</span>
            row<span class="token punctuation">.</span>clear<span class="token punctuation">(</span><span class="token punctuation">)</span>
            <span class="token keyword">continue</span>
        <span class="token keyword">if</span> index <span class="token operator">==</span> len<span class="token punctuation">(</span>self<span class="token punctuation">.</span>image_type_info<span class="token punctuation">)</span><span class="token punctuation">:</span>
            row<span class="token punctuation">.</span>extend<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token string">'-'</span><span class="token punctuation">,</span> <span class="token string">'-'</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
            table<span class="token punctuation">.</span>add_row<span class="token punctuation">(</span>row<span class="token punctuation">)</span>
    <span class="token keyword">print</span><span class="token punctuation">(</span>table<span class="token punctuation">)</span> 

init_tips_info<span class="token punctuation">(</span><span class="token punctuation">)</span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
<h4 id="2-获取用户输入"><a href="#2-获取用户输入" class="headerlink" title="2. 获取用户输入"></a>2. 获取用户输入</h4><pre class="line-numbers language-python"><code class="language-python"><span class="token keyword">while</span> <span class="token boolean">True</span><span class="token punctuation">:</span>
    image_type <span class="token operator">=</span> input<span class="token punctuation">(</span><span class="token string">'请选择下载的壁纸分类(序号,默认回车为0) >>  '</span><span class="token punctuation">)</span>
    <span class="token keyword">if</span> image_type<span class="token punctuation">:</span>
        <span class="token keyword">if</span> <span class="token operator">not</span> image_type<span class="token punctuation">.</span>isdecimal<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">or</span> int<span class="token punctuation">(</span>image_type<span class="token punctuation">)</span> <span class="token operator">not</span> <span class="token keyword">in</span> image_type_info<span class="token punctuation">:</span>
            <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">'您输入的分类序号有误，请重新确认后输入'</span><span class="token punctuation">)</span>
            <span class="token keyword">continue</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            image_type <span class="token operator">=</span> int<span class="token punctuation">(</span>image_type<span class="token punctuation">)</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        image_type <span class="token operator">=</span> <span class="token number">0</span>

    <span class="token keyword">break</span>
<span class="token keyword">while</span> <span class="token boolean">True</span><span class="token punctuation">:</span>
    count <span class="token operator">=</span> input<span class="token punctuation">(</span><span class="token string">'请输入下载数量（整数，默认为20）>> '</span><span class="token punctuation">)</span>
    <span class="token keyword">if</span> count<span class="token punctuation">:</span>
        <span class="token keyword">if</span> <span class="token operator">not</span> count<span class="token punctuation">.</span>isdecimal<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
            <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">'您输入的数量有误，必须为整数，请重新输入'</span><span class="token punctuation">)</span>
            <span class="token keyword">continue</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            count <span class="token operator">=</span> int<span class="token punctuation">(</span>count<span class="token punctuation">)</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        count <span class="token operator">=</span> <span class="token number">20</span>
    <span class="token keyword">break</span>

main<span class="token punctuation">(</span>image_type<span class="token operator">=</span>image_type<span class="token punctuation">,</span> count<span class="token operator">=</span>count<span class="token punctuation">)</span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
<h4 id="3-添加下载进度条"><a href="#3-添加下载进度条" class="headerlink" title="3. 添加下载进度条"></a>3. 添加下载进度条</h4><pre class="line-numbers language-python"><code class="language-python"><span class="token keyword">def</span> <span class="token function">main</span><span class="token punctuation">(</span>image_type<span class="token punctuation">:</span> int<span class="token punctuation">,</span> start<span class="token punctuation">:</span> int <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> count<span class="token punctuation">:</span> int <span class="token operator">=</span> <span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    url <span class="token operator">=</span> <span class="token string">'https://bird.ioliu.cn/v2'</span>

    <span class="token keyword">if</span> image_type <span class="token operator">==</span> <span class="token number">0</span><span class="token punctuation">:</span>
        params <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token string">'url'</span><span class="token punctuation">:</span> f<span class="token string">'http://wallpaper.apc.360.cn/index.php?c=WallPaper&amp;start={start}&amp;count={count}&amp;from=360chrome&amp;a=getAppsByOrder&amp;order=create_time'</span><span class="token punctuation">}</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        params <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token string">'url'</span><span class="token punctuation">:</span> f<span class="token string">'http://wallpaper.apc.360.cn/index.php?c=WallPaper&amp;start={start}&amp;count={count}&amp;from=360chrome&amp;a=getAppsByCategory&amp;cid={image_type}'</span><span class="token punctuation">}</span>

    <span class="token comment" spellcheck="true"># 图片文件夹分类</span>
    image_type_desc <span class="token operator">=</span> image_type_info<span class="token punctuation">[</span>image_type<span class="token punctuation">]</span>
    <span class="token keyword">if</span> <span class="token operator">not</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>exists<span class="token punctuation">(</span>f<span class="token string">'images/{image_type_desc}'</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        os<span class="token punctuation">.</span>makedirs<span class="token punctuation">(</span>f<span class="token string">'images/{image_type_desc}'</span><span class="token punctuation">)</span>

    <span class="token comment" spellcheck="true"># 发送请求</span>
    response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>url<span class="token operator">=</span>url<span class="token punctuation">,</span> params<span class="token operator">=</span>params<span class="token punctuation">)</span>
    result <span class="token operator">=</span> response<span class="token punctuation">.</span>json<span class="token punctuation">(</span><span class="token punctuation">)</span>

    task_progress <span class="token operator">=</span> tqdm<span class="token punctuation">(</span>result<span class="token punctuation">[</span><span class="token string">'data'</span><span class="token punctuation">]</span><span class="token punctuation">,</span> ncols<span class="token operator">=</span><span class="token number">100</span><span class="token punctuation">)</span>

    <span class="token keyword">for</span> index<span class="token punctuation">,</span> image_info <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>task_progress<span class="token punctuation">,</span> start<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        task_progress<span class="token punctuation">.</span>set_description<span class="token punctuation">(</span>f<span class="token string">"正在下载第 {index} 张图片"</span><span class="token punctuation">)</span>
        uid <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'id'</span><span class="token punctuation">]</span>
        image_url <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'url'</span><span class="token punctuation">]</span>
        tags <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'tag'</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'_category_'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'_全部_'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span>strip<span class="token punctuation">(</span><span class="token string">'_ '</span><span class="token punctuation">)</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">' '</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
        size <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'resolution'</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'x'</span><span class="token punctuation">,</span> <span class="token string">'_'</span><span class="token punctuation">)</span>
        filepath <span class="token operator">=</span> f<span class="token string">'images/{image_type_info[image_type]}/{tags}_{uid}_{size}.jpg'</span>
        response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>image_url<span class="token punctuation">)</span>
        <span class="token keyword">with</span> open<span class="token punctuation">(</span>filepath<span class="token punctuation">,</span> mode<span class="token operator">=</span><span class="token string">'wb'</span><span class="token punctuation">)</span> <span class="token keyword">as</span> f<span class="token punctuation">:</span>
            f<span class="token punctuation">.</span>write<span class="token punctuation">(</span>response<span class="token punctuation">.</span>content<span class="token punctuation">)</span>
        <span class="token comment" spellcheck="true"># print(f'{image_url} {filepath} 下载成功')</span>

    <span class="token keyword">print</span><span class="token punctuation">(</span>f<span class="token string">'所有图片下载完成!'</span><span class="token punctuation">)</span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
<h4 id="4-代码整合"><a href="#4-代码整合" class="headerlink" title="4. 代码整合"></a>4. 代码整合</h4><pre class="line-numbers language-python"><code class="language-python"><span class="token keyword">import</span> os

<span class="token keyword">import</span> requests
<span class="token keyword">from</span> prettytable <span class="token keyword">import</span> PrettyTable
<span class="token keyword">from</span> tqdm <span class="token keyword">import</span> tqdm

image_type_info <span class="token operator">=</span> <span class="token punctuation">{</span>
    <span class="token number">0</span><span class="token punctuation">:</span> <span class="token string">'最新壁纸'</span><span class="token punctuation">,</span> <span class="token number">6</span><span class="token punctuation">:</span> <span class="token string">'美女模特'</span><span class="token punctuation">,</span> <span class="token number">30</span><span class="token punctuation">:</span> <span class="token string">'爱情美图'</span><span class="token punctuation">,</span> <span class="token number">9</span><span class="token punctuation">:</span> <span class="token string">'风景大片'</span><span class="token punctuation">,</span>
    <span class="token number">36</span><span class="token punctuation">:</span> <span class="token string">'4K专区'</span><span class="token punctuation">,</span> <span class="token number">15</span><span class="token punctuation">:</span> <span class="token string">'小清新'</span><span class="token punctuation">,</span> <span class="token number">26</span><span class="token punctuation">:</span> <span class="token string">'动漫卡通'</span><span class="token punctuation">,</span> <span class="token number">11</span><span class="token punctuation">:</span> <span class="token string">'明星风尚'</span><span class="token punctuation">,</span>
    <span class="token number">14</span><span class="token punctuation">:</span> <span class="token string">'萌宠动物'</span><span class="token punctuation">,</span> <span class="token number">5</span><span class="token punctuation">:</span> <span class="token string">'游戏壁纸'</span><span class="token punctuation">,</span> <span class="token number">12</span><span class="token punctuation">:</span> <span class="token string">'汽车天下'</span><span class="token punctuation">,</span> <span class="token number">10</span><span class="token punctuation">:</span> <span class="token string">'炫酷时尚'</span><span class="token punctuation">,</span>
    <span class="token number">29</span><span class="token punctuation">:</span> <span class="token string">'月历壁纸'</span><span class="token punctuation">,</span> <span class="token number">7</span><span class="token punctuation">:</span> <span class="token string">'景视剧照'</span><span class="token punctuation">,</span> <span class="token number">35</span><span class="token punctuation">:</span> <span class="token string">'文字控'</span><span class="token punctuation">,</span> <span class="token number">13</span><span class="token punctuation">:</span> <span class="token string">'节日美图'</span><span class="token punctuation">,</span>
    <span class="token number">22</span><span class="token punctuation">:</span> <span class="token string">'军事天地'</span><span class="token punctuation">,</span> <span class="token number">16</span><span class="token punctuation">:</span> <span class="token string">'劲爆体育'</span><span class="token punctuation">,</span> <span class="token number">18</span><span class="token punctuation">:</span> <span class="token string">'BABY秀'</span>
<span class="token punctuation">}</span>

<span class="token keyword">def</span> <span class="token function">init_tips_info</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    table <span class="token operator">=</span> PrettyTable<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token string">'序号1'</span><span class="token punctuation">,</span> <span class="token string">'壁纸分类1'</span><span class="token punctuation">,</span> <span class="token string">'序号2'</span><span class="token punctuation">,</span> <span class="token string">'壁纸分类2'</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
    row <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span>
    <span class="token keyword">for</span> index<span class="token punctuation">,</span> key <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>image_type_info<span class="token punctuation">,</span> start<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        row<span class="token punctuation">.</span>extend<span class="token punctuation">(</span><span class="token punctuation">[</span>key<span class="token punctuation">,</span> image_type_info<span class="token punctuation">[</span>key<span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
        <span class="token keyword">if</span> len<span class="token punctuation">(</span>row<span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token number">4</span><span class="token punctuation">:</span>
            table<span class="token punctuation">.</span>add_row<span class="token punctuation">(</span>row<span class="token punctuation">)</span>
            row<span class="token punctuation">.</span>clear<span class="token punctuation">(</span><span class="token punctuation">)</span>
            <span class="token keyword">continue</span>
        <span class="token keyword">if</span> index <span class="token operator">==</span> len<span class="token punctuation">(</span>image_type_info<span class="token punctuation">)</span><span class="token punctuation">:</span>
            row<span class="token punctuation">.</span>extend<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token string">'-'</span><span class="token punctuation">,</span> <span class="token string">'-'</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
            table<span class="token punctuation">.</span>add_row<span class="token punctuation">(</span>row<span class="token punctuation">)</span>
    <span class="token keyword">print</span><span class="token punctuation">(</span>table<span class="token punctuation">)</span>

<span class="token keyword">def</span> <span class="token function">main</span><span class="token punctuation">(</span>image_type<span class="token punctuation">:</span> int<span class="token punctuation">,</span> start<span class="token punctuation">:</span> int <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> count<span class="token punctuation">:</span> int <span class="token operator">=</span> <span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    url <span class="token operator">=</span> <span class="token string">'https://bird.ioliu.cn/v2'</span>

    <span class="token keyword">if</span> image_type <span class="token operator">==</span> <span class="token number">0</span><span class="token punctuation">:</span>
        params <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token string">'url'</span><span class="token punctuation">:</span> f<span class="token string">'http://wallpaper.apc.360.cn/index.php?c=WallPaper&amp;start={start}&amp;count={count}&amp;from=360chrome&amp;a=getAppsByOrder&amp;order=create_time'</span><span class="token punctuation">}</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        params <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token string">'url'</span><span class="token punctuation">:</span> f<span class="token string">'http://wallpaper.apc.360.cn/index.php?c=WallPaper&amp;start={start}&amp;count={count}&amp;from=360chrome&amp;a=getAppsByCategory&amp;cid={image_type}'</span><span class="token punctuation">}</span>

    <span class="token comment" spellcheck="true"># 图片文件夹分类</span>
    image_type_desc <span class="token operator">=</span> image_type_info<span class="token punctuation">[</span>image_type<span class="token punctuation">]</span>
    <span class="token keyword">if</span> <span class="token operator">not</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>exists<span class="token punctuation">(</span>f<span class="token string">'images/{image_type_desc}'</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        os<span class="token punctuation">.</span>makedirs<span class="token punctuation">(</span>f<span class="token string">'images/{image_type_desc}'</span><span class="token punctuation">)</span>

    <span class="token comment" spellcheck="true"># 发送请求</span>
    response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>url<span class="token operator">=</span>url<span class="token punctuation">,</span> params<span class="token operator">=</span>params<span class="token punctuation">)</span>
    result <span class="token operator">=</span> response<span class="token punctuation">.</span>json<span class="token punctuation">(</span><span class="token punctuation">)</span>

    task_progress <span class="token operator">=</span> tqdm<span class="token punctuation">(</span>result<span class="token punctuation">[</span><span class="token string">'data'</span><span class="token punctuation">]</span><span class="token punctuation">,</span> ncols<span class="token operator">=</span><span class="token number">100</span><span class="token punctuation">)</span>

    <span class="token keyword">for</span> index<span class="token punctuation">,</span> image_info <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>task_progress<span class="token punctuation">,</span> start<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        task_progress<span class="token punctuation">.</span>set_description<span class="token punctuation">(</span>f<span class="token string">"正在下载第 {index} 张图片"</span><span class="token punctuation">)</span>
        uid <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'id'</span><span class="token punctuation">]</span>
        image_url <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'url'</span><span class="token punctuation">]</span>
        tags <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'tag'</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'_category_'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'_全部_'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span>strip<span class="token punctuation">(</span><span class="token string">'_ '</span><span class="token punctuation">)</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">' '</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
        size <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'resolution'</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'x'</span><span class="token punctuation">,</span> <span class="token string">'_'</span><span class="token punctuation">)</span>
        filepath <span class="token operator">=</span> f<span class="token string">'images/{image_type_info[image_type]}/{tags}_{uid}_{size}.jpg'</span>
        response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>image_url<span class="token punctuation">)</span>
        <span class="token keyword">with</span> open<span class="token punctuation">(</span>filepath<span class="token punctuation">,</span> mode<span class="token operator">=</span><span class="token string">'wb'</span><span class="token punctuation">)</span> <span class="token keyword">as</span> f<span class="token punctuation">:</span>
            f<span class="token punctuation">.</span>write<span class="token punctuation">(</span>response<span class="token punctuation">.</span>content<span class="token punctuation">)</span>
        <span class="token comment" spellcheck="true"># print(f'{image_url} {filepath} 下载成功')</span>

    <span class="token keyword">print</span><span class="token punctuation">(</span>f<span class="token string">'所有图片下载完成!'</span><span class="token punctuation">)</span>

init_tips_info<span class="token punctuation">(</span><span class="token punctuation">)</span>

<span class="token keyword">while</span> <span class="token boolean">True</span><span class="token punctuation">:</span>
    image_type <span class="token operator">=</span> input<span class="token punctuation">(</span><span class="token string">'请选择下载的壁纸分类(序号,默认回车为0) >>  '</span><span class="token punctuation">)</span>
    <span class="token keyword">if</span> image_type<span class="token punctuation">:</span>
        <span class="token keyword">if</span> <span class="token operator">not</span> image_type<span class="token punctuation">.</span>isdecimal<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">or</span> int<span class="token punctuation">(</span>image_type<span class="token punctuation">)</span> <span class="token operator">not</span> <span class="token keyword">in</span> image_type_info<span class="token punctuation">:</span>
            <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">'您输入的分类序号有误，请重新确认后输入'</span><span class="token punctuation">)</span>
            <span class="token keyword">continue</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            image_type <span class="token operator">=</span> int<span class="token punctuation">(</span>image_type<span class="token punctuation">)</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        image_type <span class="token operator">=</span> <span class="token number">0</span>

    <span class="token keyword">break</span>
<span class="token keyword">while</span> <span class="token boolean">True</span><span class="token punctuation">:</span>
    count <span class="token operator">=</span> input<span class="token punctuation">(</span><span class="token string">'请输入下载数量（整数，默认为20）>> '</span><span class="token punctuation">)</span>
    <span class="token keyword">if</span> count<span class="token punctuation">:</span>
        <span class="token keyword">if</span> <span class="token operator">not</span> count<span class="token punctuation">.</span>isdecimal<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
            <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">'您输入的数量有误，必须为整数，请重新输入'</span><span class="token punctuation">)</span>
            <span class="token keyword">continue</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            count <span class="token operator">=</span> int<span class="token punctuation">(</span>count<span class="token punctuation">)</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        count <span class="token operator">=</span> <span class="token number">20</span>
    <span class="token keyword">break</span>

main<span class="token punctuation">(</span>image_type<span class="token operator">=</span>image_type<span class="token punctuation">,</span> count<span class="token operator">=</span>count<span class="token punctuation">)</span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
<p>运行结果</p>
<pre class="line-numbers language-python"><code class="language-python"><span class="token operator">+</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">+</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">+</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">+</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">+</span>
<span class="token operator">|</span> 序号<span class="token number">1</span> <span class="token operator">|</span> 壁纸分类<span class="token number">1</span> <span class="token operator">|</span> 序号<span class="token number">2</span> <span class="token operator">|</span> 壁纸分类<span class="token number">2</span> <span class="token operator">|</span>
<span class="token operator">+</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">+</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">+</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">+</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">+</span>
<span class="token operator">|</span>   <span class="token number">0</span>   <span class="token operator">|</span>  最新壁纸 <span class="token operator">|</span>   <span class="token number">6</span>   <span class="token operator">|</span>  美女模特 <span class="token operator">|</span>
<span class="token operator">|</span>   <span class="token number">30</span>  <span class="token operator">|</span>  爱情美图 <span class="token operator">|</span>   <span class="token number">9</span>   <span class="token operator">|</span>  风景大片 <span class="token operator">|</span>
<span class="token operator">|</span>   <span class="token number">36</span>  <span class="token operator">|</span>   4K专区  <span class="token operator">|</span>   <span class="token number">15</span>  <span class="token operator">|</span>   小清新  <span class="token operator">|</span>
<span class="token operator">|</span>   <span class="token number">26</span>  <span class="token operator">|</span>  动漫卡通 <span class="token operator">|</span>   <span class="token number">11</span>  <span class="token operator">|</span>  明星风尚 <span class="token operator">|</span>
<span class="token operator">|</span>   <span class="token number">14</span>  <span class="token operator">|</span>  萌宠动物 <span class="token operator">|</span>   <span class="token number">5</span>   <span class="token operator">|</span>  游戏壁纸 <span class="token operator">|</span>
<span class="token operator">|</span>   <span class="token number">12</span>  <span class="token operator">|</span>  汽车天下 <span class="token operator">|</span>   <span class="token number">10</span>  <span class="token operator">|</span>  炫酷时尚 <span class="token operator">|</span>
<span class="token operator">|</span>   <span class="token number">29</span>  <span class="token operator">|</span>  月历壁纸 <span class="token operator">|</span>   <span class="token number">7</span>   <span class="token operator">|</span>  景视剧照 <span class="token operator">|</span>
<span class="token operator">|</span>   <span class="token number">35</span>  <span class="token operator">|</span>   文字控  <span class="token operator">|</span>   <span class="token number">13</span>  <span class="token operator">|</span>  节日美图 <span class="token operator">|</span>
<span class="token operator">|</span>   <span class="token number">22</span>  <span class="token operator">|</span>  军事天地 <span class="token operator">|</span>   <span class="token number">16</span>  <span class="token operator">|</span>  劲爆体育 <span class="token operator">|</span>
<span class="token operator">|</span>   <span class="token number">18</span>  <span class="token operator">|</span>   BABY秀  <span class="token operator">|</span>   <span class="token operator">-</span>   <span class="token operator">|</span>     <span class="token operator">-</span>     <span class="token operator">|</span>
<span class="token operator">+</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">+</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">+</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">+</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">-</span><span class="token operator">+</span>
请选择下载的壁纸分类<span class="token punctuation">(</span>序号<span class="token punctuation">,</span>默认回车为<span class="token number">0</span><span class="token punctuation">)</span> <span class="token operator">>></span>  <span class="token number">9</span>
请输入下载数量（整数，默认为<span class="token number">20</span>）<span class="token operator">>></span> <span class="token number">30</span>
正在下载第 <span class="token number">30</span> 张图片<span class="token punctuation">:</span> <span class="token number">100</span><span class="token operator">%</span><span class="token operator">|</span>█████████████████████████████████████████<span class="token operator">|</span> <span class="token number">30</span><span class="token operator">/</span><span class="token number">30</span> <span class="token punctuation">[</span><span class="token number">00</span><span class="token punctuation">:</span><span class="token number">17</span><span class="token operator">&lt;</span><span class="token number">00</span><span class="token punctuation">:</span><span class="token number">00</span><span class="token punctuation">,</span>  <span class="token number">1.</span>69it<span class="token operator">/</span>s<span class="token punctuation">]</span>
所有图片下载完成!<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
<h3 id="六、项目优化"><a href="#六、项目优化" class="headerlink" title="六、项目优化"></a>六、项目优化</h3><h4 id="1-实现过滤功能"><a href="#1-实现过滤功能" class="headerlink" title="1. 实现过滤功能"></a>1. 实现过滤功能</h4><pre class="line-numbers language-python"><code class="language-python"><span class="token keyword">def</span> <span class="token function">init_history_file</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    history_file_path <span class="token operator">=</span> <span class="token string">'history.txt'</span>
    history_file <span class="token operator">=</span> open<span class="token punctuation">(</span>history_file_path<span class="token punctuation">,</span> mode<span class="token operator">=</span><span class="token string">'a+'</span><span class="token punctuation">,</span> encoding<span class="token operator">=</span><span class="token string">'utf-8'</span><span class="token punctuation">)</span>
    downloaded_images <span class="token operator">=</span> set<span class="token punctuation">(</span><span class="token punctuation">)</span>

    <span class="token keyword">if</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>exists<span class="token punctuation">(</span>history_file_path<span class="token punctuation">)</span><span class="token punctuation">:</span>
        history_file<span class="token punctuation">.</span>seek<span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">)</span>
        downloaded_images <span class="token operator">=</span> <span class="token punctuation">{</span>line<span class="token punctuation">.</span>strip<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">for</span> line <span class="token keyword">in</span> history_file<span class="token punctuation">}</span>
    <span class="token keyword">return</span> history_file<span class="token punctuation">,</span> downloaded_images

<span class="token keyword">def</span> <span class="token function">main</span><span class="token punctuation">(</span>image_type<span class="token punctuation">:</span> int<span class="token punctuation">,</span> start<span class="token punctuation">:</span> int <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> count<span class="token punctuation">:</span> int <span class="token operator">=</span> <span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    history_file<span class="token punctuation">,</span> downloaded_images <span class="token operator">=</span> init_history_file<span class="token punctuation">(</span><span class="token punctuation">)</span>
    url <span class="token operator">=</span> <span class="token string">'https://bird.ioliu.cn/v2'</span>
    <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span>
    task_progress <span class="token operator">=</span> tqdm<span class="token punctuation">(</span>result<span class="token punctuation">[</span><span class="token string">'data'</span><span class="token punctuation">]</span><span class="token punctuation">,</span> ncols<span class="token operator">=</span><span class="token number">100</span><span class="token punctuation">)</span>

    <span class="token keyword">for</span> index<span class="token punctuation">,</span> image_info <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>task_progress<span class="token punctuation">,</span> start<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        task_progress<span class="token punctuation">.</span>set_description<span class="token punctuation">(</span>f<span class="token string">"正在下载第 {index} 张图片"</span><span class="token punctuation">)</span>
        uid <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'id'</span><span class="token punctuation">]</span>
        image_url <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'url'</span><span class="token punctuation">]</span>
        tags <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'tag'</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'_category_'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'_全部_'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span>strip<span class="token punctuation">(</span><span class="token string">'_ '</span><span class="token punctuation">)</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">' '</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
        size <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'resolution'</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'x'</span><span class="token punctuation">,</span> <span class="token string">'_'</span><span class="token punctuation">)</span>
        filepath <span class="token operator">=</span> f<span class="token string">'images/{image_type_info[image_type]}/{tags}_{uid}_{size}.jpg'</span>
        <span class="token keyword">if</span> filepath <span class="token keyword">in</span> downloaded_images<span class="token punctuation">:</span>
            <span class="token keyword">print</span><span class="token punctuation">(</span>f<span class="token string">'{filepath} 下载过了'</span><span class="token punctuation">)</span>
            <span class="token keyword">continue</span>
        response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>image_url<span class="token punctuation">)</span>
        <span class="token keyword">with</span> open<span class="token punctuation">(</span>filepath<span class="token punctuation">,</span> mode<span class="token operator">=</span><span class="token string">'wb'</span><span class="token punctuation">)</span> <span class="token keyword">as</span> f<span class="token punctuation">:</span>
            f<span class="token punctuation">.</span>write<span class="token punctuation">(</span>response<span class="token punctuation">.</span>content<span class="token punctuation">)</span>
        history_file<span class="token punctuation">.</span>write<span class="token punctuation">(</span>f<span class="token string">'{filepath}\n'</span><span class="token punctuation">)</span>

    <span class="token keyword">print</span><span class="token punctuation">(</span>f<span class="token string">'所有图片下载完成!'</span><span class="token punctuation">)</span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
<p>第一次运行</p>
<pre><code>+-------+-----------+-------+-----------+
| 序号1 | 壁纸分类1 | 序号2 | 壁纸分类2 |
+-------+-----------+-------+-----------+
|   0   |  最新壁纸 |   6   |  美女模特 |
|   30  |  爱情美图 |   9   |  风景大片 |
|   36  |   4K专区  |   15  |   小清新  |
|   26  |  动漫卡通 |   11  |  明星风尚 |
|   14  |  萌宠动物 |   5   |  游戏壁纸 |
|   12  |  汽车天下 |   10  |  炫酷时尚 |
|   29  |  月历壁纸 |   7   |  景视剧照 |
|   35  |   文字控  |   13  |  节日美图 |
|   22  |  军事天地 |   16  |  劲爆体育 |
|   18  |   BABY秀  |   -   |     -     |
+-------+-----------+-------+-----------+
请选择下载的壁纸分类(序号,默认回车为0) &gt;&gt;  9
请输入下载数量（整数，默认为20）&gt;&gt; 30
正在下载第 30 张图片: 100%|█████████████████████████████████████████| 30/30 [00:13&lt;00:00,  2.30it/s]
所有图片下载完成!</code></pre>
<p>查看图片文件夹</p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210406115311249.png"></p>
<p>当前目录下生成了一个history.txt文件，文件内容</p>
<pre><code>images/风景大片/海洋天堂_海岸_沙滩_黄昏_风景大片_2046860_3840_2160.jpg
images/风景大片/乌云_麦田_树_平原_旷野_风景大片_2046785_1920_1080.jpg
images/风景大片/秋意正浓_乡村_原野_深秋_风景大片_2046775_1920_1080.jpg
...
images/风景大片/秋意正浓_雪山_高原_树林_神秘_风景大片_2045072_1920_1080.jpg
images/风景大片/海洋天堂_度假_浪漫_海边_沙滩_吊床_风景大片_2045010_3840_2160.jpg
</code></pre>
<p>第二次运行</p>
<pre><code>+-------+-----------+-------+-----------+
| 序号1 | 壁纸分类1 | 序号2 | 壁纸分类2 |
+-------+-----------+-------+-----------+
|   0   |  最新壁纸 |   6   |  美女模特 |
|   30  |  爱情美图 |   9   |  风景大片 |
|   36  |   4K专区  |   15  |   小清新  |
|   26  |  动漫卡通 |   11  |  明星风尚 |
|   14  |  萌宠动物 |   5   |  游戏壁纸 |
|   12  |  汽车天下 |   10  |  炫酷时尚 |
|   29  |  月历壁纸 |   7   |  景视剧照 |
|   35  |   文字控  |   13  |  节日美图 |
|   22  |  军事天地 |   16  |  劲爆体育 |
|   18  |   BABY秀  |   -   |     -     |
+-------+-----------+-------+-----------+
请选择下载的壁纸分类(序号,默认回车为0) &gt;&gt;  9
请输入下载数量（整数，默认为20）&gt;&gt; 30
images/风景大片/海洋天堂_海岸_沙滩_黄昏_风景大片_2046860_3840_2160.jpg 下载过了
正在下载第 30 张图片: 100%|███████████████████████████████████████| 30/30 [00:00&lt;00:00, 3004.95it/s]
images/风景大片/乌云_麦田_树_平原_旷野_风景大片_2046785_1920_1080.jpg 下载过了
images/风景大片/秋意正浓_乡村_原野_深秋_风景大片_2046775_1920_1080.jpg 下载过了
images/风景大片/自然风光_荒漠_石头_傍晚_风景大片_2046626_2560_1440.jpg 下载过了
...
images/风景大片/海洋天堂_度假_浪漫_海边_沙滩_吊床_风景大片_2045010_3840_2160.jpg 下载过了
所有图片下载完成!</code></pre>
<h4 id="2-多线程下载加速"><a href="#2-多线程下载加速" class="headerlink" title="2. 多线程下载加速"></a>2. 多线程下载加速</h4><pre class="line-numbers language-python"><code class="language-python"><span class="token keyword">from</span> concurrent<span class="token punctuation">.</span>futures <span class="token keyword">import</span> ProcessPoolExecutor

<span class="token keyword">def</span> <span class="token function">search_images</span><span class="token punctuation">(</span>image_type<span class="token punctuation">:</span> int<span class="token punctuation">,</span> start<span class="token punctuation">:</span> int <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> count<span class="token punctuation">:</span> int <span class="token operator">=</span> <span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">print</span><span class="token punctuation">(</span>f<span class="token string">'正在检索 - {image_type_info[image_type]}'</span><span class="token punctuation">)</span>
    url <span class="token operator">=</span> <span class="token string">'https://bird.ioliu.cn/v2'</span>
    <span class="token keyword">if</span> image_type <span class="token operator">==</span> <span class="token number">0</span><span class="token punctuation">:</span>
        params <span class="token operator">=</span> <span class="token punctuation">{</span>
            <span class="token string">'url'</span><span class="token punctuation">:</span> f<span class="token string">'http://wallpaper.apc.360.cn/index.php?c=WallPaper&amp;start={start}&amp;count={count}&amp;from=360chrome&amp;a=getAppsByOrder&amp;order=create_time'</span><span class="token punctuation">}</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        params <span class="token operator">=</span> <span class="token punctuation">{</span>
            <span class="token string">'url'</span><span class="token punctuation">:</span> f<span class="token string">'http://wallpaper.apc.360.cn/index.php?c=WallPaper&amp;start={start}&amp;count={count}&amp;from=360chrome&amp;a=getAppsByCategory&amp;cid={image_type}'</span><span class="token punctuation">}</span>

    <span class="token comment" spellcheck="true"># 图片文件夹分类</span>
    image_type_desc <span class="token operator">=</span> image_type_info<span class="token punctuation">[</span>image_type<span class="token punctuation">]</span>
    <span class="token keyword">if</span> <span class="token operator">not</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>exists<span class="token punctuation">(</span>f<span class="token string">'images/{image_type_desc}'</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        os<span class="token punctuation">.</span>makedirs<span class="token punctuation">(</span>f<span class="token string">'images/{image_type_desc}'</span><span class="token punctuation">)</span>

    <span class="token comment" spellcheck="true"># 发送请求</span>
    response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>url<span class="token operator">=</span>url<span class="token punctuation">,</span> params<span class="token operator">=</span>params<span class="token punctuation">)</span>
    result <span class="token operator">=</span> response<span class="token punctuation">.</span>json<span class="token punctuation">(</span><span class="token punctuation">)</span>
    downloaded_count <span class="token operator">=</span> <span class="token number">0</span>
    <span class="token keyword">for</span> image_info <span class="token keyword">in</span> result<span class="token punctuation">[</span><span class="token string">'data'</span><span class="token punctuation">]</span><span class="token punctuation">:</span>
        uid <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'id'</span><span class="token punctuation">]</span>
        image_url <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'url'</span><span class="token punctuation">]</span>
        tags <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'tag'</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'_category_'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'_全部_'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span>strip<span class="token punctuation">(</span><span class="token string">'_ '</span><span class="token punctuation">)</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">' '</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
        size <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'resolution'</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'x'</span><span class="token punctuation">,</span> <span class="token string">'_'</span><span class="token punctuation">)</span>
        filepath <span class="token operator">=</span> f<span class="token string">'images/{image_type_info[image_type]}/{tags}_{uid}_{size}.jpg'</span>
        <span class="token keyword">if</span> filepath <span class="token operator">not</span> <span class="token keyword">in</span> downloaded_images<span class="token punctuation">:</span>
            image_data_list<span class="token punctuation">.</span>add<span class="token punctuation">(</span><span class="token punctuation">(</span>image_url<span class="token punctuation">,</span> filepath<span class="token punctuation">)</span><span class="token punctuation">)</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            downloaded_count <span class="token operator">+=</span> <span class="token number">1</span>
    <span class="token keyword">return</span> image_data_list<span class="token punctuation">,</span> downloaded_count


<span class="token keyword">def</span> <span class="token function">start_download</span><span class="token punctuation">(</span>image_data_list<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">if</span> len<span class="token punctuation">(</span>image_data_list<span class="token punctuation">)</span> <span class="token operator">></span> <span class="token number">30</span><span class="token punctuation">:</span>
        choice <span class="token operator">=</span> input<span class="token punctuation">(</span><span class="token string">'是否开启多进程加速下载？(Y/N) >> '</span><span class="token punctuation">)</span>
        <span class="token keyword">if</span> choice<span class="token punctuation">.</span>upper<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">'Y'</span><span class="token punctuation">:</span>
            data_list <span class="token operator">=</span> <span class="token punctuation">[</span>set<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">for</span> _ <span class="token keyword">in</span> range<span class="token punctuation">(</span><span class="token number">4</span><span class="token punctuation">)</span><span class="token punctuation">]</span>
            <span class="token keyword">for</span> index<span class="token punctuation">,</span> image <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>image_data_list<span class="token punctuation">)</span><span class="token punctuation">:</span>
                remainder <span class="token operator">=</span> index <span class="token operator">%</span> <span class="token number">4</span>
                data_list<span class="token punctuation">[</span>remainder<span class="token punctuation">]</span><span class="token punctuation">.</span>add<span class="token punctuation">(</span>image<span class="token punctuation">)</span>
            <span class="token keyword">with</span> ProcessPoolExecutor<span class="token punctuation">(</span>max_workers<span class="token operator">=</span><span class="token number">4</span><span class="token punctuation">)</span> <span class="token keyword">as</span> pool<span class="token punctuation">:</span>
                pool<span class="token punctuation">.</span>map<span class="token punctuation">(</span>download_images<span class="token punctuation">,</span> data_list<span class="token punctuation">)</span>
            <span class="token keyword">print</span><span class="token punctuation">(</span>f<span class="token string">'\n\n所有图片下载完成!'</span><span class="token punctuation">)</span>
            <span class="token keyword">return</span>
    download_images<span class="token punctuation">(</span><span class="token punctuation">)</span>
    <span class="token keyword">print</span><span class="token punctuation">(</span>f<span class="token string">'所有图片下载完成!'</span><span class="token punctuation">)</span>


<span class="token keyword">def</span> <span class="token function">download_images</span><span class="token punctuation">(</span>data_list<span class="token punctuation">:</span> set <span class="token operator">=</span> None<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">if</span> data_list<span class="token punctuation">:</span>
        task_progress <span class="token operator">=</span> tqdm<span class="token punctuation">(</span>data_list<span class="token punctuation">,</span> ncols<span class="token operator">=</span><span class="token number">100</span><span class="token punctuation">)</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        task_progress <span class="token operator">=</span> tqdm<span class="token punctuation">(</span>image_data_list<span class="token punctuation">,</span> ncols<span class="token operator">=</span><span class="token number">100</span><span class="token punctuation">)</span>
    <span class="token keyword">for</span> index<span class="token punctuation">,</span> task <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>task_progress<span class="token punctuation">,</span> start<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        url<span class="token punctuation">,</span> filepath <span class="token operator">=</span> task
        task_progress<span class="token punctuation">.</span>set_description<span class="token punctuation">(</span>f<span class="token string">"正在下载第 {index} 张图片"</span><span class="token punctuation">)</span>
        response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>url<span class="token operator">=</span>url<span class="token punctuation">)</span>
        <span class="token keyword">with</span> open<span class="token punctuation">(</span>filepath<span class="token punctuation">,</span> mode<span class="token operator">=</span><span class="token string">'wb'</span><span class="token punctuation">)</span> <span class="token keyword">as</span> f<span class="token punctuation">:</span>
            f<span class="token punctuation">.</span>write<span class="token punctuation">(</span>response<span class="token punctuation">.</span>content<span class="token punctuation">)</span>
        history_file<span class="token punctuation">.</span>write<span class="token punctuation">(</span>f<span class="token string">'{filepath}\n'</span><span class="token punctuation">)</span>


<span class="token keyword">def</span> <span class="token function">run</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    init_tips_info<span class="token punctuation">(</span><span class="token punctuation">)</span>
    <span class="token keyword">while</span> <span class="token boolean">True</span><span class="token punctuation">:</span>
        image_type <span class="token operator">=</span> input<span class="token punctuation">(</span><span class="token string">'请选择下载的壁纸分类(序号,默认回车为0) >>  '</span><span class="token punctuation">)</span>
        <span class="token keyword">if</span> image_type<span class="token punctuation">:</span>
            <span class="token keyword">if</span> <span class="token operator">not</span> image_type<span class="token punctuation">.</span>isdecimal<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">or</span> int<span class="token punctuation">(</span>image_type<span class="token punctuation">)</span> <span class="token operator">not</span> <span class="token keyword">in</span> image_type_info<span class="token punctuation">:</span>
                <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">'您输入的分类序号有误，请重新确认后输入'</span><span class="token punctuation">)</span>
                <span class="token keyword">continue</span>
            <span class="token keyword">else</span><span class="token punctuation">:</span>
                image_type <span class="token operator">=</span> int<span class="token punctuation">(</span>image_type<span class="token punctuation">)</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            image_type <span class="token operator">=</span> <span class="token number">0</span>

        <span class="token keyword">break</span>
    <span class="token keyword">while</span> <span class="token boolean">True</span><span class="token punctuation">:</span>
        count <span class="token operator">=</span> input<span class="token punctuation">(</span><span class="token string">'请输入下载数量（整数，默认为20）>> '</span><span class="token punctuation">)</span>
        <span class="token keyword">if</span> count<span class="token punctuation">:</span>
            <span class="token keyword">if</span> <span class="token operator">not</span> count<span class="token punctuation">.</span>isdecimal<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
                <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">'您输入的数量有误，必须为整数，请重新输入'</span><span class="token punctuation">)</span>
                <span class="token keyword">continue</span>
            <span class="token keyword">else</span><span class="token punctuation">:</span>
                count <span class="token operator">=</span> int<span class="token punctuation">(</span>count<span class="token punctuation">)</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            count <span class="token operator">=</span> <span class="token number">20</span>
        <span class="token keyword">break</span>

    image_data_list<span class="token punctuation">,</span> downloaded_count <span class="token operator">=</span> search_images<span class="token punctuation">(</span>image_type<span class="token operator">=</span>image_type<span class="token punctuation">,</span> count<span class="token operator">=</span>count<span class="token punctuation">)</span>

    <span class="token keyword">if</span> downloaded_count <span class="token operator">==</span> <span class="token number">0</span><span class="token punctuation">:</span>
        <span class="token keyword">print</span><span class="token punctuation">(</span>f<span class="token string">'共检索到 {len(image_data_list)} 张图片, 开始下载'</span><span class="token punctuation">)</span>
    <span class="token keyword">elif</span> len<span class="token punctuation">(</span>image_data_list<span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token number">0</span><span class="token punctuation">:</span>
        <span class="token keyword">print</span><span class="token punctuation">(</span>f<span class="token string">'共检索到 {downloaded_count} 张图片, 已全部下载完成！'</span><span class="token punctuation">)</span>
        <span class="token keyword">return</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        <span class="token keyword">print</span><span class="token punctuation">(</span>
            f<span class="token string">'共检索到 {len(image_data_list) + downloaded_count} 张图片, 已下载 {downloaded_count} 张, 还需下载 {len(image_data_list)} 张, 开始下载'</span><span class="token punctuation">)</span>

    start_download<span class="token punctuation">(</span>image_data_list<span class="token operator">=</span>image_data_list<span class="token punctuation">)</span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
<p>运行代码</p>
<p>第一次运行</p>
<pre><code>+-------+-----------+-------+-----------+
| 序号1 | 壁纸分类1 | 序号2 | 壁纸分类2 |
+-------+-----------+-------+-----------+
|   0   |  最新壁纸 |   6   |  美女模特 |
|   30  |  爱情美图 |   9   |  风景大片 |
|   36  |   4K专区  |   15  |   小清新  |
|   26  |  动漫卡通 |   11  |  明星风尚 |
|   14  |  萌宠动物 |   5   |  游戏壁纸 |
|   12  |  汽车天下 |   10  |  炫酷时尚 |
|   29  |  月历壁纸 |   7   |  景视剧照 |
|   35  |   文字控  |   13  |  节日美图 |
|   22  |  军事天地 |   16  |  劲爆体育 |
|   18  |   BABY秀  |   -   |     -     |
+-------+-----------+-------+-----------+
请选择下载的壁纸分类(序号,默认回车为0) &gt;&gt;  9
请输入下载数量（整数，默认为20）&gt;&gt; 50
正在检索 - 风景大片
共检索到 50 张图片, 开始下载
是否选择多线程下载？(Y/N) &gt;&gt; y
正在下载第 12 张图片: 100%|█████████████████████████████████████████| 12/12 [00:03&lt;00:00,  3.57it/s]
正在下载第 12 张图片: 100%|█████████████████████████████████████████| 12/12 [00:03&lt;00:00,  3.19it/s]
正在下载第 13 张图片: 100%|█████████████████████████████████████████| 13/13 [00:04&lt;00:00,  3.09it/s]
正在下载第 13 张图片: 100%|█████████████████████████████████████████| 13/13 [00:04&lt;00:00,  3.01it/s]
所有图片下载完成!</code></pre>
<p>第二次</p>
<pre><code>+-------+-----------+-------+-----------+
| 序号1 | 壁纸分类1 | 序号2 | 壁纸分类2 |
+-------+-----------+-------+-----------+
|   0   |  最新壁纸 |   6   |  美女模特 |
|   30  |  爱情美图 |   9   |  风景大片 |
|   36  |   4K专区  |   15  |   小清新  |
|   26  |  动漫卡通 |   11  |  明星风尚 |
|   14  |  萌宠动物 |   5   |  游戏壁纸 |
|   12  |  汽车天下 |   10  |  炫酷时尚 |
|   29  |  月历壁纸 |   7   |  景视剧照 |
|   35  |   文字控  |   13  |  节日美图 |
|   22  |  军事天地 |   16  |  劲爆体育 |
|   18  |   BABY秀  |   -   |     -     |
+-------+-----------+-------+-----------+
请选择下载的壁纸分类(序号,默认回车为0) &gt;&gt;  9
请输入下载数量（整数，默认为20）&gt;&gt; 50
正在检索 - 风景大片
共检索到 50 张图片, 已全部下载完成！</code></pre>
<h4 id="3-为打印输出添加色彩"><a href="#3-为打印输出添加色彩" class="headerlink" title="3. 为打印输出添加色彩"></a>3. 为打印输出添加色彩</h4><p><code>utils.py</code></p>
<pre class="line-numbers language-python"><code class="language-python"><span class="token keyword">class</span> <span class="token class-name">Foreground</span><span class="token punctuation">:</span>
    Black <span class="token operator">=</span> <span class="token number">30</span>
    Red <span class="token operator">=</span> <span class="token number">31</span>
    Green <span class="token operator">=</span> <span class="token number">32</span>
    Yellow <span class="token operator">=</span> <span class="token number">33</span>
    Blue <span class="token operator">=</span> <span class="token number">34</span>
    Purplish_red <span class="token operator">=</span> <span class="token number">35</span>
    Cyan <span class="token operator">=</span> <span class="token number">36</span>
    White <span class="token operator">=</span> <span class="token number">37</span>


<span class="token keyword">class</span> <span class="token class-name">Background</span><span class="token punctuation">:</span>
    Black <span class="token operator">=</span> <span class="token number">40</span>
    Red <span class="token operator">=</span> <span class="token number">41</span>
    Green <span class="token operator">=</span> <span class="token number">42</span>
    Yellow <span class="token operator">=</span> <span class="token number">43</span>
    Blue <span class="token operator">=</span> <span class="token number">44</span>
    Purplish_red <span class="token operator">=</span> <span class="token number">45</span>
    Cyan <span class="token operator">=</span> <span class="token number">46</span>
    White <span class="token operator">=</span> <span class="token number">47</span>


<span class="token keyword">class</span> <span class="token class-name">Display</span><span class="token punctuation">:</span>
    Default <span class="token operator">=</span> <span class="token number">0</span>
    Highlight <span class="token operator">=</span> <span class="token number">1</span>
    Underline <span class="token operator">=</span> <span class="token number">4</span>
    Twinkle <span class="token operator">=</span> <span class="token number">5</span>
    Reverse <span class="token operator">=</span> <span class="token number">7</span>
    Invisible <span class="token operator">=</span> <span class="token number">8</span>


<span class="token keyword">def</span> <span class="token function">print</span><span class="token punctuation">(</span>string<span class="token punctuation">:</span> str<span class="token punctuation">,</span> display_mode<span class="token punctuation">:</span> int <span class="token operator">=</span> Display<span class="token punctuation">.</span>Default<span class="token punctuation">,</span> foreground<span class="token punctuation">:</span> int <span class="token operator">=</span> Foreground<span class="token punctuation">.</span>Green<span class="token punctuation">,</span>
                background<span class="token punctuation">:</span> int <span class="token operator">=</span> Background<span class="token punctuation">.</span>Black<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">print</span><span class="token punctuation">(</span>f<span class="token string">"\033[{display_mode};{foreground};{background}m{string}\033[0m"</span><span class="token punctuation">)</span>


<span class="token keyword">def</span> <span class="token function">add_color</span><span class="token punctuation">(</span>string<span class="token punctuation">:</span> str<span class="token punctuation">,</span> display_mode<span class="token punctuation">:</span> int <span class="token operator">=</span> Display<span class="token punctuation">.</span>Default<span class="token punctuation">,</span> foreground<span class="token punctuation">:</span> int <span class="token operator">=</span> Foreground<span class="token punctuation">.</span>Green<span class="token punctuation">,</span>
              background<span class="token punctuation">:</span> int <span class="token operator">=</span> Background<span class="token punctuation">.</span>Black<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">return</span> f<span class="token string">"\033[{display_mode};{foreground};{background}m{string}\033[0m"</span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
<p><code>spider.py</code></p>
<pre class="line-numbers language-python"><code class="language-python"><span class="token keyword">import</span> os
<span class="token keyword">from</span> concurrent<span class="token punctuation">.</span>futures <span class="token keyword">import</span> ProcessPoolExecutor

<span class="token keyword">import</span> requests
<span class="token keyword">from</span> prettytable <span class="token keyword">import</span> PrettyTable
<span class="token keyword">from</span> tqdm <span class="token keyword">import</span> tqdm

<span class="token keyword">from</span> utils <span class="token keyword">import</span> print_color

image_type_info <span class="token operator">=</span> <span class="token punctuation">{</span>
    <span class="token number">0</span><span class="token punctuation">:</span> <span class="token string">'最新壁纸'</span><span class="token punctuation">,</span> <span class="token number">6</span><span class="token punctuation">:</span> <span class="token string">'美女模特'</span><span class="token punctuation">,</span> <span class="token number">30</span><span class="token punctuation">:</span> <span class="token string">'爱情美图'</span><span class="token punctuation">,</span> <span class="token number">9</span><span class="token punctuation">:</span> <span class="token string">'风景大片'</span><span class="token punctuation">,</span>
    <span class="token number">36</span><span class="token punctuation">:</span> <span class="token string">'4K专区'</span><span class="token punctuation">,</span> <span class="token number">15</span><span class="token punctuation">:</span> <span class="token string">'小清新'</span><span class="token punctuation">,</span> <span class="token number">26</span><span class="token punctuation">:</span> <span class="token string">'动漫卡通'</span><span class="token punctuation">,</span> <span class="token number">11</span><span class="token punctuation">:</span> <span class="token string">'明星风尚'</span><span class="token punctuation">,</span>
    <span class="token number">14</span><span class="token punctuation">:</span> <span class="token string">'萌宠动物'</span><span class="token punctuation">,</span> <span class="token number">5</span><span class="token punctuation">:</span> <span class="token string">'游戏壁纸'</span><span class="token punctuation">,</span> <span class="token number">12</span><span class="token punctuation">:</span> <span class="token string">'汽车天下'</span><span class="token punctuation">,</span> <span class="token number">10</span><span class="token punctuation">:</span> <span class="token string">'炫酷时尚'</span><span class="token punctuation">,</span>
    <span class="token number">29</span><span class="token punctuation">:</span> <span class="token string">'月历壁纸'</span><span class="token punctuation">,</span> <span class="token number">7</span><span class="token punctuation">:</span> <span class="token string">'景视剧照'</span><span class="token punctuation">,</span> <span class="token number">35</span><span class="token punctuation">:</span> <span class="token string">'文字控'</span><span class="token punctuation">,</span> <span class="token number">13</span><span class="token punctuation">:</span> <span class="token string">'节日美图'</span><span class="token punctuation">,</span>
    <span class="token number">22</span><span class="token punctuation">:</span> <span class="token string">'军事天地'</span><span class="token punctuation">,</span> <span class="token number">16</span><span class="token punctuation">:</span> <span class="token string">'劲爆体育'</span><span class="token punctuation">,</span> <span class="token number">18</span><span class="token punctuation">:</span> <span class="token string">'BABY秀'</span>
<span class="token punctuation">}</span>

image_data_list <span class="token operator">=</span> set<span class="token punctuation">(</span><span class="token punctuation">)</span>
history_file_path <span class="token operator">=</span> <span class="token string">'history.txt'</span>
history_file <span class="token operator">=</span> open<span class="token punctuation">(</span>history_file_path<span class="token punctuation">,</span> mode<span class="token operator">=</span><span class="token string">'a+'</span><span class="token punctuation">,</span> encoding<span class="token operator">=</span><span class="token string">'utf-8'</span><span class="token punctuation">)</span>
downloaded_images <span class="token operator">=</span> set<span class="token punctuation">(</span><span class="token punctuation">)</span>
<span class="token keyword">if</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>exists<span class="token punctuation">(</span>history_file_path<span class="token punctuation">)</span><span class="token punctuation">:</span>
    history_file<span class="token punctuation">.</span>seek<span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">)</span>
    downloaded_images <span class="token operator">=</span> <span class="token punctuation">{</span>line<span class="token punctuation">.</span>strip<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">for</span> line <span class="token keyword">in</span> history_file<span class="token punctuation">}</span>


<span class="token keyword">def</span> <span class="token function">init_tips_info</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    table <span class="token operator">=</span> PrettyTable<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token string">'序号1'</span><span class="token punctuation">,</span> <span class="token string">'壁纸分类1'</span><span class="token punctuation">,</span> <span class="token string">'序号2'</span><span class="token punctuation">,</span> <span class="token string">'壁纸分类2'</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
    row <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span>
    <span class="token keyword">for</span> index<span class="token punctuation">,</span> key <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>image_type_info<span class="token punctuation">,</span> start<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        row<span class="token punctuation">.</span>extend<span class="token punctuation">(</span><span class="token punctuation">[</span>key<span class="token punctuation">,</span> image_type_info<span class="token punctuation">[</span>key<span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
        <span class="token keyword">if</span> len<span class="token punctuation">(</span>row<span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token number">4</span><span class="token punctuation">:</span>
            table<span class="token punctuation">.</span>add_row<span class="token punctuation">(</span>row<span class="token punctuation">)</span>
            row<span class="token punctuation">.</span>clear<span class="token punctuation">(</span><span class="token punctuation">)</span>
            <span class="token keyword">continue</span>
        <span class="token keyword">if</span> index <span class="token operator">==</span> len<span class="token punctuation">(</span>image_type_info<span class="token punctuation">)</span><span class="token punctuation">:</span>
            row<span class="token punctuation">.</span>extend<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token string">'-'</span><span class="token punctuation">,</span> <span class="token string">'-'</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
            table<span class="token punctuation">.</span>add_row<span class="token punctuation">(</span>row<span class="token punctuation">)</span>
    <span class="token keyword">print</span><span class="token punctuation">(</span>table<span class="token punctuation">)</span>


<span class="token keyword">def</span> <span class="token function">search_images</span><span class="token punctuation">(</span>image_type<span class="token punctuation">:</span> int<span class="token punctuation">,</span> start<span class="token punctuation">:</span> int <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> count<span class="token punctuation">:</span> int <span class="token operator">=</span> <span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    print_color<span class="token punctuation">(</span>f<span class="token string">'正在检索 - {image_type_info[image_type]}'</span><span class="token punctuation">)</span>
    url <span class="token operator">=</span> <span class="token string">'https://bird.ioliu.cn/v2'</span>
    <span class="token keyword">if</span> image_type <span class="token operator">==</span> <span class="token number">0</span><span class="token punctuation">:</span>
        params <span class="token operator">=</span> <span class="token punctuation">{</span>
            <span class="token string">'url'</span><span class="token punctuation">:</span> f<span class="token string">'http://wallpaper.apc.360.cn/index.php?c=WallPaper&amp;start={start}&amp;count={count}&amp;from=360chrome&amp;a=getAppsByOrder&amp;order=create_time'</span><span class="token punctuation">}</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        params <span class="token operator">=</span> <span class="token punctuation">{</span>
            <span class="token string">'url'</span><span class="token punctuation">:</span> f<span class="token string">'http://wallpaper.apc.360.cn/index.php?c=WallPaper&amp;start={start}&amp;count={count}&amp;from=360chrome&amp;a=getAppsByCategory&amp;cid={image_type}'</span><span class="token punctuation">}</span>

    <span class="token comment" spellcheck="true"># 图片文件夹分类</span>
    image_type_desc <span class="token operator">=</span> image_type_info<span class="token punctuation">[</span>image_type<span class="token punctuation">]</span>
    <span class="token keyword">if</span> <span class="token operator">not</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>exists<span class="token punctuation">(</span>f<span class="token string">'images/{image_type_desc}'</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        os<span class="token punctuation">.</span>makedirs<span class="token punctuation">(</span>f<span class="token string">'images/{image_type_desc}'</span><span class="token punctuation">)</span>

    <span class="token comment" spellcheck="true"># 发送请求</span>
    response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>url<span class="token operator">=</span>url<span class="token punctuation">,</span> params<span class="token operator">=</span>params<span class="token punctuation">)</span>
    result <span class="token operator">=</span> response<span class="token punctuation">.</span>json<span class="token punctuation">(</span><span class="token punctuation">)</span>
    downloaded_count <span class="token operator">=</span> <span class="token number">0</span>
    <span class="token keyword">for</span> image_info <span class="token keyword">in</span> result<span class="token punctuation">[</span><span class="token string">'data'</span><span class="token punctuation">]</span><span class="token punctuation">:</span>
        uid <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'id'</span><span class="token punctuation">]</span>
        image_url <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'url'</span><span class="token punctuation">]</span>
        tags <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'tag'</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'_category_'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'_全部_'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span>strip<span class="token punctuation">(</span><span class="token string">'_ '</span><span class="token punctuation">)</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">' '</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
        size <span class="token operator">=</span> image_info<span class="token punctuation">[</span><span class="token string">'resolution'</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'x'</span><span class="token punctuation">,</span> <span class="token string">'_'</span><span class="token punctuation">)</span>
        filepath <span class="token operator">=</span> f<span class="token string">'images/{image_type_info[image_type]}/{tags}_{uid}_{size}.jpg'</span>
        <span class="token keyword">if</span> filepath <span class="token operator">not</span> <span class="token keyword">in</span> downloaded_images<span class="token punctuation">:</span>
            image_data_list<span class="token punctuation">.</span>add<span class="token punctuation">(</span><span class="token punctuation">(</span>image_url<span class="token punctuation">,</span> filepath<span class="token punctuation">)</span><span class="token punctuation">)</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            downloaded_count <span class="token operator">+=</span> <span class="token number">1</span>
    <span class="token keyword">return</span> image_data_list<span class="token punctuation">,</span> downloaded_count


<span class="token keyword">def</span> <span class="token function">start_download</span><span class="token punctuation">(</span>image_data_list<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">if</span> len<span class="token punctuation">(</span>image_data_list<span class="token punctuation">)</span> <span class="token operator">></span> <span class="token number">30</span><span class="token punctuation">:</span>
        choice <span class="token operator">=</span> input<span class="token punctuation">(</span><span class="token string">'是否开启多进程加速下载？(Y/N) >> '</span><span class="token punctuation">)</span>
        <span class="token keyword">if</span> choice<span class="token punctuation">.</span>upper<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">'Y'</span><span class="token punctuation">:</span>
            data_list <span class="token operator">=</span> <span class="token punctuation">[</span>set<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">for</span> _ <span class="token keyword">in</span> range<span class="token punctuation">(</span><span class="token number">4</span><span class="token punctuation">)</span><span class="token punctuation">]</span>
            <span class="token keyword">for</span> index<span class="token punctuation">,</span> image <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>image_data_list<span class="token punctuation">)</span><span class="token punctuation">:</span>
                remainder <span class="token operator">=</span> index <span class="token operator">%</span> <span class="token number">4</span>
                data_list<span class="token punctuation">[</span>remainder<span class="token punctuation">]</span><span class="token punctuation">.</span>add<span class="token punctuation">(</span>image<span class="token punctuation">)</span>
            <span class="token keyword">with</span> ProcessPoolExecutor<span class="token punctuation">(</span>max_workers<span class="token operator">=</span><span class="token number">4</span><span class="token punctuation">)</span> <span class="token keyword">as</span> pool<span class="token punctuation">:</span>
                pool<span class="token punctuation">.</span>map<span class="token punctuation">(</span>download_images<span class="token punctuation">,</span> data_list<span class="token punctuation">)</span>
            print_color<span class="token punctuation">(</span>f<span class="token string">'\n\n所有图片下载完成!'</span><span class="token punctuation">)</span>
            <span class="token keyword">return</span>
    download_images<span class="token punctuation">(</span><span class="token punctuation">)</span>
    print_color<span class="token punctuation">(</span>f<span class="token string">'所有图片下载完成!'</span><span class="token punctuation">)</span>


<span class="token keyword">def</span> <span class="token function">download_images</span><span class="token punctuation">(</span>data_list<span class="token punctuation">:</span> set <span class="token operator">=</span> None<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">if</span> data_list<span class="token punctuation">:</span>
        task_progress <span class="token operator">=</span> tqdm<span class="token punctuation">(</span>data_list<span class="token punctuation">,</span> ncols<span class="token operator">=</span><span class="token number">100</span><span class="token punctuation">)</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        task_progress <span class="token operator">=</span> tqdm<span class="token punctuation">(</span>image_data_list<span class="token punctuation">,</span> ncols<span class="token operator">=</span><span class="token number">100</span><span class="token punctuation">)</span>
    <span class="token keyword">for</span> index<span class="token punctuation">,</span> task <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>task_progress<span class="token punctuation">,</span> start<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        url<span class="token punctuation">,</span> filepath <span class="token operator">=</span> task
        task_progress<span class="token punctuation">.</span>set_description<span class="token punctuation">(</span>f<span class="token string">"正在下载第 {index} 张图片"</span><span class="token punctuation">)</span>
        response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>url<span class="token operator">=</span>url<span class="token punctuation">)</span>
        <span class="token keyword">with</span> open<span class="token punctuation">(</span>filepath<span class="token punctuation">,</span> mode<span class="token operator">=</span><span class="token string">'wb'</span><span class="token punctuation">)</span> <span class="token keyword">as</span> f<span class="token punctuation">:</span>
            f<span class="token punctuation">.</span>write<span class="token punctuation">(</span>response<span class="token punctuation">.</span>content<span class="token punctuation">)</span>
        history_file<span class="token punctuation">.</span>write<span class="token punctuation">(</span>f<span class="token string">'{filepath}\n'</span><span class="token punctuation">)</span>


<span class="token keyword">def</span> <span class="token function">run</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    init_tips_info<span class="token punctuation">(</span><span class="token punctuation">)</span>
    <span class="token keyword">while</span> <span class="token boolean">True</span><span class="token punctuation">:</span>
        image_type <span class="token operator">=</span> input<span class="token punctuation">(</span><span class="token string">'请选择下载的壁纸分类(序号,默认回车为0) >>  '</span><span class="token punctuation">)</span>
        <span class="token keyword">if</span> image_type<span class="token punctuation">:</span>
            <span class="token keyword">if</span> <span class="token operator">not</span> image_type<span class="token punctuation">.</span>isdecimal<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">or</span> int<span class="token punctuation">(</span>image_type<span class="token punctuation">)</span> <span class="token operator">not</span> <span class="token keyword">in</span> image_type_info<span class="token punctuation">:</span>
                <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">'您输入的分类序号有误，请重新确认后输入'</span><span class="token punctuation">)</span>
                <span class="token keyword">continue</span>
            <span class="token keyword">else</span><span class="token punctuation">:</span>
                image_type <span class="token operator">=</span> int<span class="token punctuation">(</span>image_type<span class="token punctuation">)</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            image_type <span class="token operator">=</span> <span class="token number">0</span>

        <span class="token keyword">break</span>
    <span class="token keyword">while</span> <span class="token boolean">True</span><span class="token punctuation">:</span>
        count <span class="token operator">=</span> input<span class="token punctuation">(</span><span class="token string">'请输入下载数量（整数，默认为20）>> '</span><span class="token punctuation">)</span>
        <span class="token keyword">if</span> count<span class="token punctuation">:</span>
            <span class="token keyword">if</span> <span class="token operator">not</span> count<span class="token punctuation">.</span>isdecimal<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
                <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">'您输入的数量有误，必须为整数，请重新输入'</span><span class="token punctuation">)</span>
                <span class="token keyword">continue</span>
            <span class="token keyword">else</span><span class="token punctuation">:</span>
                count <span class="token operator">=</span> int<span class="token punctuation">(</span>count<span class="token punctuation">)</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            count <span class="token operator">=</span> <span class="token number">20</span>
        <span class="token keyword">break</span>

    image_data_list<span class="token punctuation">,</span> downloaded_count <span class="token operator">=</span> search_images<span class="token punctuation">(</span>image_type<span class="token operator">=</span>image_type<span class="token punctuation">,</span> count<span class="token operator">=</span>count<span class="token punctuation">)</span>

    <span class="token keyword">if</span> downloaded_count <span class="token operator">==</span> <span class="token number">0</span><span class="token punctuation">:</span>
        print_color<span class="token punctuation">(</span>f<span class="token string">'共检索到 {len(image_data_list)} 张图片, 开始下载'</span><span class="token punctuation">)</span>
    <span class="token keyword">elif</span> len<span class="token punctuation">(</span>image_data_list<span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token number">0</span><span class="token punctuation">:</span>
        print_color<span class="token punctuation">(</span>f<span class="token string">'共检索到 {downloaded_count} 张图片, 已全部下载完成！'</span><span class="token punctuation">)</span>
        <span class="token keyword">return</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        print_color<span class="token punctuation">(</span>
            f<span class="token string">'共检索到 {len(image_data_list) + downloaded_count} 张图片, 已下载 {downloaded_count} 张, 还需下载 {len(image_data_list)} 张, 开始下载'</span><span class="token punctuation">)</span>

    start_download<span class="token punctuation">(</span>image_data_list<span class="token operator">=</span>image_data_list<span class="token punctuation">)</span>


<span class="token keyword">if</span> __name__ <span class="token operator">==</span> <span class="token string">'__main__'</span><span class="token punctuation">:</span>
    run<span class="token punctuation">(</span><span class="token punctuation">)</span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
<p>运行结果</p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210406170545949.png"></p>
<p>运行代码演示</p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/spider_netnr.gif"></p>
<h4 id="4-面向对象"><a href="#4-面向对象" class="headerlink" title="4. 面向对象"></a>4. 面向对象</h4><p>以上代码均是基于函数式编程书写的，虽然目前代码总体只有150行左右，但是能感觉到随着功能的扩展，模块之后的位置和关联还是有些乱的，所以有必要将其改写成面向对象的形式了。代码我已经写好，有兴趣的伙伴可以进行学习：<a target="_blank" rel="noopener" href="https://gitee.com/zhangyafeii/python-doc-tutorial/tree/master/code/%E7%BD%91%E7%BB%9C%E7%88%AC%E8%99%AB/%E7%88%AC%E8%99%AB%E5%AE%9E%E6%88%98%EF%BC%9A%E9%AB%98%E6%B8%85%E5%9B%BE%E7%89%87%E4%B8%8B%E8%BD%BD">爬虫实战：高清图片下载</a></p>
<h2 id="项目二：彼岸图网"><a href="#项目二：彼岸图网" class="headerlink" title="项目二：彼岸图网"></a>项目二：彼岸图网</h2><h3 id="一、目标网站分析-1"><a href="#一、目标网站分析-1" class="headerlink" title="一、目标网站分析"></a>一、目标网站分析</h3><p>网址：<a target="_blank" rel="noopener" href="https://pic.netbian.com/">https://pic.netbian.com/</a></p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210406173924641.png"></p>
<p>该网站提供了4K风景、4K美女等12种壁纸类别，用户可以选择任一类别的图片进行查看和下载。点击某一分类可以发现，浏览器地址栏的url追加了对应的分类路径，观察分类下的图片，鼠标右键可以直接下载图片，下载之后发现分辨率较低，不太清晰，我们点击进去任意一张图片，查看详细内容页面，发现详细页面的图片相较于检索页分辨率高了不少，另外还可以下载原图，不过原图要登录和收费，所以暂时我们不考虑，详细页面显示图片的分辨率已经相对很好了。</p>
<p>我们的目标就是自动化获取和下载详细页面的图片。</p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210406174126569.png"></p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210406174301838.png"></p>
<h3 id="二、网络请求分析-1"><a href="#二、网络请求分析-1" class="headerlink" title="二、网络请求分析"></a>二、网络请求分析</h3><p>鼠标右键点击检查，刷新页面观察浏览器请求变化，发现该页面地址栏请求为GET请求，响应数据为HTML，里面有我们需要的图片详细页面url信息，通过点击该链接，进入详细页面，详细页面可以获取到我们需要的图片url。</p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210406175307706.png"></p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210406175423634.png"></p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210406175747238.png"></p>
<h3 id="三、初步实现-1"><a href="#三、初步实现-1" class="headerlink" title="三、初步实现"></a>三、初步实现</h3><ul>
<li><p>代码</p>
<pre class="line-numbers language-python"><code class="language-python"><span class="token keyword">from</span> urllib<span class="token punctuation">.</span>parse <span class="token keyword">import</span> urljoin

<span class="token keyword">from</span> lxml <span class="token keyword">import</span> etree

<span class="token keyword">import</span> requests

url <span class="token operator">=</span> <span class="token string">'https://pic.netbian.com'</span>

response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>url<span class="token punctuation">)</span>
html <span class="token operator">=</span> etree<span class="token punctuation">.</span>HTML<span class="token punctuation">(</span>response<span class="token punctuation">.</span>content<span class="token punctuation">)</span>

<span class="token keyword">for</span> li <span class="token keyword">in</span> html<span class="token punctuation">.</span>xpath<span class="token punctuation">(</span><span class="token string">'//*[@id="main"]/div[3]/ul/li'</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    href <span class="token operator">=</span> li<span class="token punctuation">.</span>xpath<span class="token punctuation">(</span><span class="token string">'a/@href'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span>
    uid <span class="token operator">=</span> href<span class="token punctuation">.</span>rsplit<span class="token punctuation">(</span><span class="token string">'/'</span><span class="token punctuation">,</span> maxsplit<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'.html'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
    title <span class="token operator">=</span> li<span class="token punctuation">.</span>xpath<span class="token punctuation">(</span><span class="token string">'a//img/@alt'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">' '</span><span class="token punctuation">,</span> <span class="token string">'_'</span><span class="token punctuation">)</span>
    response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>urljoin<span class="token punctuation">(</span>url<span class="token punctuation">,</span> href<span class="token punctuation">)</span><span class="token punctuation">)</span>
    html <span class="token operator">=</span> etree<span class="token punctuation">.</span>HTML<span class="token punctuation">(</span>response<span class="token punctuation">.</span>content<span class="token punctuation">)</span>
    img_url <span class="token operator">=</span> urljoin<span class="token punctuation">(</span>url<span class="token punctuation">,</span> html<span class="token punctuation">.</span>xpath<span class="token punctuation">(</span><span class="token string">'//a[@id="img"]/img/@src'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
    <span class="token keyword">print</span><span class="token punctuation">(</span>uid<span class="token punctuation">,</span> title<span class="token punctuation">,</span> img_url<span class="token punctuation">)</span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
</li>
<li><p>运行结果</p>
<pre><code>27188 海_少女_大鱼_女孩_唯美动漫4k壁纸3840x2160 https://pic.netbian.com/uploads/allimg/210405/224731-16176340512348.jpg
24695 下午_趴在桌子的女孩4k动漫壁纸3840x2160 https://pic.netbian.com/uploads/allimg/190824/212516-15666531161ade.jpg
27180 学生_女子_丝袜_美腿_二次元动漫美女4k壁纸 https://pic.netbian.com/uploads/allimg/210403/002739-1617380859fd5c.jpg
27074 肉骨茶_个性动漫美女_骨汤屋_二次元美女_4k动漫壁纸 https://pic.netbian.com/uploads/allimg/210318/135851-16160471316886.jpg
27160 陆萱萱_黑色丝袜美腿美女_白色衬衫_背带裤_4k美女壁纸 https://pic.netbian.com/uploads/allimg/210401/220258-16172857783909.jpg
26434 陆萱萱_白色衬衫_黑色吊带短裤_黑丝美腿美女4k壁纸 https://pic.netbian.com/uploads/allimg/201008/203736-1602160656f945.jpg
27190 LOL英雄联盟damwon_gaming_冠军皮肤_卡牌_凯南_蕾欧娜_奈德丽_烬4k壁纸 https://pic.netbian.com/uploads/allimg/210405/225104-1617634264a669.jpg
25331 女生_女子_起床_晚上_都市_夜景_4k动漫壁纸 https://pic.netbian.com/uploads/allimg/200102/193708-15779650287a6a.jpg
21953 阿尔卑斯山风景4k高清壁纸3840x2160 https://pic.netbian.com/uploads/allimg/180826/113958-153525479855be.jpg
26010 长发少女黑色吊带裙_好看的4k动漫美女壁纸3840x2160 https://pic.netbian.com/uploads/allimg/200618/005100-1592412660d6f4.jpg
17781 糖果_美女模特4k壁纸 https://pic.netbian.com/uploads/allimg/180128/112234-1517109754d925.jpg
26903 床_女孩_黑色睡衣_小乌龟_唯美插画_4k动漫壁纸 https://pic.netbian.com/uploads/allimg/210216/205419-1613480059a058.jpg
27146 古风美女_眼泪_伞_告别_唯美意境_插画_4k动漫壁纸 https://pic.netbian.com/uploads/allimg/210328/001020-1616861420d950.jpg
26780 古风_美少女_伞_长发_女孩大长腿4k唯美动漫壁纸 https://pic.netbian.com/uploads/allimg/210122/195550-1611316550d711.jpg
26123 漂亮美少女_女孩子_手机_白裤袜_美脚美腿_好看二次元4k动漫壁纸 https://pic.netbian.com/uploads/allimg/200714/224033-159473763328ff.jpg
25716 dva_宅女_长筒袜_短发_可爱女孩4k壁纸 https://pic.netbian.com/uploads/allimg/200410/213246-1586525566c099.jpg
26309 克拉女神高清女神江琴_唯美写真摄影4k电脑壁纸 https://pic.netbian.com/uploads/allimg/200910/200207-15997393279b23.jpg
24031 女孩_微笑_蹲着_水_海岸_海滩_波浪_风_夜晚_4k动漫壁纸 https://pic.netbian.com/uploads/allimg/190415/214606-15553359663cd8.jpg
25688 女孩子_旅行_车窗_火车_列车_听音乐_高清4k动漫壁纸 https://pic.netbian.com/uploads/allimg/200405/171302-15860779827765.jpg
25037 车子_少女_女生_绑辫子_唯美好看4k动漫壁纸 https://pic.netbian.com/uploads/allimg/191026/001458-157202009858e9.jpg</code></pre>
<p>通过以上代码，我们实现了任务的核心功能，同时验证了我们之前分析的合理性。</p>
</li>
</ul>
<h3 id="四、功能完善和优化"><a href="#四、功能完善和优化" class="headerlink" title="四、功能完善和优化"></a>四、功能完善和优化</h3><p>有了上一个案例的基础，我们可以仿照实现相似的功能。</p>
<h4 id="版本1"><a href="#版本1" class="headerlink" title="版本1"></a>版本1</h4><ul>
<li><p>代码</p>
<p>utils.py</p>
<pre class="line-numbers language-python"><code class="language-python"><span class="token keyword">class</span> <span class="token class-name">Foreground</span><span class="token punctuation">:</span>
    Black <span class="token operator">=</span> <span class="token number">30</span>
    Red <span class="token operator">=</span> <span class="token number">31</span>
    Green <span class="token operator">=</span> <span class="token number">32</span>
    Yellow <span class="token operator">=</span> <span class="token number">33</span>
    Blue <span class="token operator">=</span> <span class="token number">34</span>
    Purplish_red <span class="token operator">=</span> <span class="token number">35</span>
    Cyan <span class="token operator">=</span> <span class="token number">36</span>
    White <span class="token operator">=</span> <span class="token number">37</span>

<span class="token keyword">class</span> <span class="token class-name">Background</span><span class="token punctuation">:</span>
    Black <span class="token operator">=</span> <span class="token number">40</span>
    Red <span class="token operator">=</span> <span class="token number">41</span>
    Green <span class="token operator">=</span> <span class="token number">42</span>
    Yellow <span class="token operator">=</span> <span class="token number">43</span>
    Blue <span class="token operator">=</span> <span class="token number">44</span>
    Purplish_red <span class="token operator">=</span> <span class="token number">45</span>
    Cyan <span class="token operator">=</span> <span class="token number">46</span>
    White <span class="token operator">=</span> <span class="token number">47</span>

<span class="token keyword">class</span> <span class="token class-name">Display</span><span class="token punctuation">:</span>
    Default <span class="token operator">=</span> <span class="token number">0</span>
    Highlight <span class="token operator">=</span> <span class="token number">1</span>
    Underline <span class="token operator">=</span> <span class="token number">4</span>
    Twinkle <span class="token operator">=</span> <span class="token number">5</span>
    Reverse <span class="token operator">=</span> <span class="token number">7</span>
    Invisible <span class="token operator">=</span> <span class="token number">8</span>

<span class="token keyword">def</span> <span class="token function">print_color</span><span class="token punctuation">(</span>string<span class="token punctuation">:</span> str<span class="token punctuation">,</span> display_mode<span class="token punctuation">:</span> int <span class="token operator">=</span> Display<span class="token punctuation">.</span>Default<span class="token punctuation">,</span> foreground<span class="token punctuation">:</span> int <span class="token operator">=</span> Foreground<span class="token punctuation">.</span>Green<span class="token punctuation">,</span>
                background<span class="token punctuation">:</span> int <span class="token operator">=</span> Background<span class="token punctuation">.</span>Black<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">print</span><span class="token punctuation">(</span>f<span class="token string">"\033[{display_mode};{foreground};{background}m{string}\033[0m"</span><span class="token punctuation">)</span>

<span class="token keyword">def</span> <span class="token function">add_color</span><span class="token punctuation">(</span>string<span class="token punctuation">:</span> str<span class="token punctuation">,</span> display_mode<span class="token punctuation">:</span> int <span class="token operator">=</span> Display<span class="token punctuation">.</span>Default<span class="token punctuation">,</span> foreground<span class="token punctuation">:</span> int <span class="token operator">=</span> Foreground<span class="token punctuation">.</span>Green<span class="token punctuation">,</span>
              background<span class="token punctuation">:</span> int <span class="token operator">=</span> Background<span class="token punctuation">.</span>Black<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">return</span> f<span class="token string">"\033[{display_mode};{foreground};{background}m{string}\033[0m"</span>
<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
<p>spider.py</p>
<pre class="line-numbers language-python"><code class="language-python"><span class="token keyword">import</span> os
<span class="token keyword">from</span> concurrent<span class="token punctuation">.</span>futures<span class="token punctuation">.</span>thread <span class="token keyword">import</span> ThreadPoolExecutor
<span class="token keyword">from</span> urllib<span class="token punctuation">.</span>parse <span class="token keyword">import</span> urljoin

<span class="token keyword">import</span> requests
<span class="token keyword">from</span> lxml <span class="token keyword">import</span> etree
<span class="token keyword">from</span> prettytable <span class="token keyword">import</span> PrettyTable
<span class="token keyword">from</span> tqdm <span class="token keyword">import</span> tqdm
<span class="token keyword">from</span> zyf<span class="token punctuation">.</span>timer <span class="token keyword">import</span> timeit

<span class="token keyword">from</span> utils <span class="token keyword">import</span> print_color

image_dir <span class="token operator">=</span> <span class="token string">'images'</span>

image_type_info <span class="token operator">=</span> <span class="token punctuation">{</span>
    <span class="token number">1</span><span class="token punctuation">:</span> <span class="token string">'4kfengjing'</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">:</span> <span class="token string">'4kmeinv'</span><span class="token punctuation">,</span> <span class="token number">3</span><span class="token punctuation">:</span> <span class="token string">'4kyouxi'</span><span class="token punctuation">,</span> <span class="token number">4</span><span class="token punctuation">:</span> <span class="token string">'4kdongman'</span><span class="token punctuation">,</span>
    <span class="token number">5</span><span class="token punctuation">:</span> <span class="token string">'4kyingshi'</span><span class="token punctuation">,</span> <span class="token number">6</span><span class="token punctuation">:</span> <span class="token string">'4kmingxing'</span><span class="token punctuation">,</span> <span class="token number">7</span><span class="token punctuation">:</span> <span class="token string">'4kqiche'</span><span class="token punctuation">,</span> <span class="token number">8</span><span class="token punctuation">:</span> <span class="token string">'4kdongwu'</span><span class="token punctuation">,</span>
    <span class="token number">9</span><span class="token punctuation">:</span> <span class="token string">'4krenwu'</span><span class="token punctuation">,</span> <span class="token number">10</span><span class="token punctuation">:</span> <span class="token string">'4kmeishi'</span><span class="token punctuation">,</span> <span class="token number">11</span><span class="token punctuation">:</span> <span class="token string">'4kzongjiao'</span><span class="token punctuation">,</span> <span class="token number">12</span><span class="token punctuation">:</span> <span class="token string">'4kbeijing'</span><span class="token punctuation">,</span>
<span class="token punctuation">}</span>
image_tip_info <span class="token operator">=</span> <span class="token punctuation">{</span>
    <span class="token number">1</span><span class="token punctuation">:</span> <span class="token string">'4K风景'</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">:</span> <span class="token string">'4K美女'</span><span class="token punctuation">,</span> <span class="token number">3</span><span class="token punctuation">:</span> <span class="token string">'4K游戏'</span><span class="token punctuation">,</span> <span class="token number">4</span><span class="token punctuation">:</span> <span class="token string">'4K动漫'</span><span class="token punctuation">,</span>
    <span class="token number">5</span><span class="token punctuation">:</span> <span class="token string">'4K影视'</span><span class="token punctuation">,</span> <span class="token number">6</span><span class="token punctuation">:</span> <span class="token string">'4K明星'</span><span class="token punctuation">,</span> <span class="token number">7</span><span class="token punctuation">:</span> <span class="token string">'4K汽车'</span><span class="token punctuation">,</span> <span class="token number">8</span><span class="token punctuation">:</span> <span class="token string">'4K动物'</span><span class="token punctuation">,</span>
    <span class="token number">9</span><span class="token punctuation">:</span> <span class="token string">'4K人物'</span><span class="token punctuation">,</span> <span class="token number">10</span><span class="token punctuation">:</span> <span class="token string">'4K美食'</span><span class="token punctuation">,</span> <span class="token number">11</span><span class="token punctuation">:</span> <span class="token string">'4K宗教'</span><span class="token punctuation">,</span> <span class="token number">12</span><span class="token punctuation">:</span> <span class="token string">'4K背景'</span><span class="token punctuation">,</span>
<span class="token punctuation">}</span>

image_data_list <span class="token operator">=</span> set<span class="token punctuation">(</span><span class="token punctuation">)</span>
search_count <span class="token operator">=</span> <span class="token number">0</span>
downloaded_count <span class="token operator">=</span> <span class="token number">0</span>
success_count <span class="token operator">=</span> <span class="token number">0</span>
history_file_path <span class="token operator">=</span> <span class="token string">'history.txt'</span>

<span class="token keyword">if</span> <span class="token operator">not</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>exists<span class="token punctuation">(</span>image_dir<span class="token punctuation">)</span><span class="token punctuation">:</span>
    os<span class="token punctuation">.</span>makedirs<span class="token punctuation">(</span>image_dir<span class="token punctuation">)</span>

history_file <span class="token operator">=</span> open<span class="token punctuation">(</span>history_file_path<span class="token punctuation">,</span> mode<span class="token operator">=</span><span class="token string">'a+'</span><span class="token punctuation">,</span> encoding<span class="token operator">=</span><span class="token string">'utf-8'</span><span class="token punctuation">)</span>
downloaded_images <span class="token operator">=</span> set<span class="token punctuation">(</span><span class="token punctuation">)</span>
<span class="token keyword">if</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>exists<span class="token punctuation">(</span>history_file_path<span class="token punctuation">)</span><span class="token punctuation">:</span>
    history_file<span class="token punctuation">.</span>seek<span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">)</span>
    downloaded_images <span class="token operator">=</span> <span class="token punctuation">{</span>line<span class="token punctuation">.</span>strip<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">for</span> line <span class="token keyword">in</span> history_file<span class="token punctuation">}</span>

<span class="token keyword">def</span> <span class="token function">init_tips_info</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    table <span class="token operator">=</span> PrettyTable<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token string">'序号1'</span><span class="token punctuation">,</span> <span class="token string">'壁纸分类1'</span><span class="token punctuation">,</span> <span class="token string">'序号2'</span><span class="token punctuation">,</span> <span class="token string">'壁纸分类2'</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
    row <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span>
    <span class="token keyword">for</span> index<span class="token punctuation">,</span> key <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>image_tip_info<span class="token punctuation">,</span> start<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        row<span class="token punctuation">.</span>extend<span class="token punctuation">(</span><span class="token punctuation">[</span>key<span class="token punctuation">,</span> image_tip_info<span class="token punctuation">[</span>key<span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
        <span class="token keyword">if</span> len<span class="token punctuation">(</span>row<span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token number">4</span><span class="token punctuation">:</span>
            table<span class="token punctuation">.</span>add_row<span class="token punctuation">(</span>row<span class="token punctuation">)</span>
            row<span class="token punctuation">.</span>clear<span class="token punctuation">(</span><span class="token punctuation">)</span>
            <span class="token keyword">continue</span>
        <span class="token keyword">if</span> index <span class="token operator">==</span> len<span class="token punctuation">(</span>image_type_info<span class="token punctuation">)</span><span class="token punctuation">:</span>
            row<span class="token punctuation">.</span>extend<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token string">'-'</span><span class="token punctuation">,</span> <span class="token string">'-'</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
            table<span class="token punctuation">.</span>add_row<span class="token punctuation">(</span>row<span class="token punctuation">)</span>
    <span class="token keyword">print</span><span class="token punctuation">(</span>table<span class="token punctuation">)</span>

<span class="token keyword">def</span> <span class="token function">search_images</span><span class="token punctuation">(</span>image_type<span class="token punctuation">:</span> int<span class="token punctuation">,</span> page_nums<span class="token punctuation">:</span> int<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">print</span><span class="token punctuation">(</span>f<span class="token string">'正在检索 - {image_tip_info[image_type]} - 请稍等 - 马上开始下载'</span><span class="token punctuation">)</span>

    url <span class="token operator">=</span> <span class="token string">'https://pic.netbian.com'</span>
    <span class="token keyword">if</span> image_type <span class="token operator">!=</span> <span class="token number">0</span><span class="token punctuation">:</span>
        url <span class="token operator">=</span> urljoin<span class="token punctuation">(</span>url<span class="token punctuation">,</span> image_type_info<span class="token punctuation">[</span>image_type<span class="token punctuation">]</span><span class="token punctuation">)</span>
        image_type_desc <span class="token operator">=</span> image_tip_info<span class="token punctuation">[</span>image_type<span class="token punctuation">]</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        url <span class="token operator">=</span> url
        image_type_desc <span class="token operator">=</span> <span class="token string">'最新壁纸'</span>

    <span class="token keyword">if</span> <span class="token operator">not</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>exists<span class="token punctuation">(</span>f<span class="token string">'{image_dir}/{image_type_desc}'</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        os<span class="token punctuation">.</span>mkdir<span class="token punctuation">(</span>f<span class="token string">'{image_dir}/{image_type_desc}'</span><span class="token punctuation">)</span>

    <span class="token keyword">if</span> page_nums <span class="token operator">></span> <span class="token number">1</span><span class="token punctuation">:</span>
        <span class="token keyword">with</span> ThreadPoolExecutor<span class="token punctuation">(</span>max_workers<span class="token operator">=</span>page_nums<span class="token punctuation">)</span> <span class="token keyword">as</span> pool<span class="token punctuation">:</span>
            pool<span class="token punctuation">.</span>map<span class="token punctuation">(</span>get_image_data<span class="token punctuation">,</span> <span class="token punctuation">[</span>url <span class="token keyword">for</span> _ <span class="token keyword">in</span> range<span class="token punctuation">(</span>page_nums<span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token punctuation">[</span>image_type_desc <span class="token keyword">for</span> _ <span class="token keyword">in</span> range<span class="token punctuation">(</span>page_nums<span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">,</span>
                     range<span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> page_nums <span class="token operator">+</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        get_image_data<span class="token punctuation">(</span>url<span class="token operator">=</span>url<span class="token punctuation">,</span> image_type_desc<span class="token operator">=</span>image_type_desc<span class="token punctuation">,</span> page_num<span class="token operator">=</span>page_nums<span class="token punctuation">)</span>

<span class="token keyword">def</span> <span class="token function">get_image_data</span><span class="token punctuation">(</span>url<span class="token punctuation">:</span> str<span class="token punctuation">,</span> image_type_desc<span class="token punctuation">:</span> str<span class="token punctuation">,</span> page_num<span class="token punctuation">:</span> int<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">global</span> downloaded_count<span class="token punctuation">,</span> search_count
    url <span class="token operator">=</span> url <span class="token keyword">if</span> page_num <span class="token operator">==</span> <span class="token number">1</span> <span class="token keyword">else</span> f<span class="token string">'{url}/index_{page_num}.html'</span>
    response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>url<span class="token punctuation">)</span>
    html <span class="token operator">=</span> etree<span class="token punctuation">.</span>HTML<span class="token punctuation">(</span>response<span class="token punctuation">.</span>content<span class="token punctuation">)</span>
    <span class="token keyword">for</span> li <span class="token keyword">in</span> html<span class="token punctuation">.</span>xpath<span class="token punctuation">(</span><span class="token string">'//*[@id="main"]/div[3]/ul/li'</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        href <span class="token operator">=</span> li<span class="token punctuation">.</span>xpath<span class="token punctuation">(</span><span class="token string">'a/@href'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span>
        uid <span class="token operator">=</span> href<span class="token punctuation">.</span>rsplit<span class="token punctuation">(</span><span class="token string">'/'</span><span class="token punctuation">,</span> maxsplit<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'.html'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
        title <span class="token operator">=</span> li<span class="token punctuation">.</span>xpath<span class="token punctuation">(</span><span class="token string">'a//img/@alt'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">' '</span><span class="token punctuation">,</span> <span class="token string">'_'</span><span class="token punctuation">)</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">','</span><span class="token punctuation">,</span> <span class="token string">'-'</span><span class="token punctuation">)</span>
        response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>urljoin<span class="token punctuation">(</span>url<span class="token punctuation">,</span> href<span class="token punctuation">)</span><span class="token punctuation">)</span>
        html <span class="token operator">=</span> etree<span class="token punctuation">.</span>HTML<span class="token punctuation">(</span>response<span class="token punctuation">.</span>content<span class="token punctuation">)</span>
        img_url <span class="token operator">=</span> urljoin<span class="token punctuation">(</span>url<span class="token punctuation">,</span> html<span class="token punctuation">.</span>xpath<span class="token punctuation">(</span><span class="token string">'//a[@id="img"]/img/@src'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
        filepath <span class="token operator">=</span> f<span class="token string">'{image_dir}/{image_type_desc}/{title}_{uid}.jpg'</span>
        <span class="token keyword">if</span> filepath <span class="token operator">not</span> <span class="token keyword">in</span> downloaded_images<span class="token punctuation">:</span>
            search_count <span class="token operator">+=</span> <span class="token number">1</span>
            image_data_list<span class="token punctuation">.</span>add<span class="token punctuation">(</span><span class="token punctuation">(</span>img_url<span class="token punctuation">,</span> filepath<span class="token punctuation">)</span><span class="token punctuation">)</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            downloaded_count <span class="token operator">+=</span> <span class="token number">1</span>

@timeit
<span class="token keyword">def</span> <span class="token function">start</span><span class="token punctuation">(</span>image_type<span class="token punctuation">:</span> int<span class="token punctuation">,</span> page_nums<span class="token punctuation">:</span> int<span class="token punctuation">)</span><span class="token punctuation">:</span>
    search_images<span class="token punctuation">(</span>image_type<span class="token operator">=</span>image_type<span class="token punctuation">,</span> page_nums<span class="token operator">=</span>page_nums<span class="token punctuation">)</span>
    start_download<span class="token punctuation">(</span>page_nums<span class="token punctuation">)</span>

<span class="token keyword">def</span> <span class="token function">start_download</span><span class="token punctuation">(</span>workers<span class="token punctuation">:</span> int<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">if</span> len<span class="token punctuation">(</span>image_data_list<span class="token punctuation">)</span> <span class="token operator">></span> <span class="token number">20</span><span class="token punctuation">:</span>
        data_list <span class="token operator">=</span> <span class="token punctuation">[</span>set<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">for</span> _ <span class="token keyword">in</span> range<span class="token punctuation">(</span>workers<span class="token punctuation">)</span><span class="token punctuation">]</span>
        <span class="token keyword">for</span> index<span class="token punctuation">,</span> image <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>image_data_list<span class="token punctuation">)</span><span class="token punctuation">:</span>
            remainder <span class="token operator">=</span> index <span class="token operator">%</span> workers
            data_list<span class="token punctuation">[</span>remainder<span class="token punctuation">]</span><span class="token punctuation">.</span>add<span class="token punctuation">(</span>image<span class="token punctuation">)</span>
        <span class="token keyword">with</span> ThreadPoolExecutor<span class="token punctuation">(</span>max_workers<span class="token operator">=</span>workers<span class="token punctuation">)</span> <span class="token keyword">as</span> pool<span class="token punctuation">:</span>
            pool<span class="token punctuation">.</span>map<span class="token punctuation">(</span>download_images<span class="token punctuation">,</span> data_list<span class="token punctuation">)</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        download_images<span class="token punctuation">(</span>image_data_list<span class="token punctuation">)</span>

<span class="token keyword">def</span> <span class="token function">download_images</span><span class="token punctuation">(</span>data_list<span class="token punctuation">:</span> set <span class="token operator">=</span> None<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">global</span> success_count
    <span class="token keyword">if</span> <span class="token operator">not</span> data_list<span class="token punctuation">:</span>
        <span class="token keyword">return</span>

    task_progress <span class="token operator">=</span> tqdm<span class="token punctuation">(</span>data_list<span class="token punctuation">,</span> ncols<span class="token operator">=</span><span class="token number">100</span><span class="token punctuation">)</span>
    <span class="token keyword">for</span> index<span class="token punctuation">,</span> task <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>task_progress<span class="token punctuation">,</span> start<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        url<span class="token punctuation">,</span> filepath <span class="token operator">=</span> task
        task_progress<span class="token punctuation">.</span>set_description<span class="token punctuation">(</span>f<span class="token string">"正在下载第 {index} 张图片"</span><span class="token punctuation">)</span>
        response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>url<span class="token operator">=</span>url<span class="token punctuation">)</span>
        <span class="token keyword">with</span> open<span class="token punctuation">(</span>filepath<span class="token punctuation">,</span> mode<span class="token operator">=</span><span class="token string">'wb'</span><span class="token punctuation">)</span> <span class="token keyword">as</span> f<span class="token punctuation">:</span>
            f<span class="token punctuation">.</span>write<span class="token punctuation">(</span>response<span class="token punctuation">.</span>content<span class="token punctuation">)</span>
        history_file<span class="token punctuation">.</span>write<span class="token punctuation">(</span>f<span class="token string">'{filepath}\n'</span><span class="token punctuation">)</span>
        success_count <span class="token operator">+=</span> <span class="token number">1</span>

<span class="token keyword">def</span> <span class="token function">run</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    init_tips_info<span class="token punctuation">(</span><span class="token punctuation">)</span>
    <span class="token keyword">while</span> <span class="token boolean">True</span><span class="token punctuation">:</span>
        image_type <span class="token operator">=</span> input<span class="token punctuation">(</span><span class="token string">'请选择下载的壁纸分类(序号,默认为最新壁纸) >>  '</span><span class="token punctuation">)</span>
        <span class="token keyword">if</span> image_type<span class="token punctuation">:</span>
            <span class="token keyword">if</span> <span class="token operator">not</span> image_type<span class="token punctuation">.</span>isdecimal<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">or</span> int<span class="token punctuation">(</span>image_type<span class="token punctuation">)</span> <span class="token operator">not</span> <span class="token keyword">in</span> image_type_info<span class="token punctuation">:</span>
                <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">'您输入的分类序号有误，请重新确认后输入'</span><span class="token punctuation">)</span>
                <span class="token keyword">continue</span>
            <span class="token keyword">else</span><span class="token punctuation">:</span>
                image_type <span class="token operator">=</span> int<span class="token punctuation">(</span>image_type<span class="token punctuation">)</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            image_type <span class="token operator">=</span> <span class="token number">0</span>
        <span class="token keyword">break</span>
    <span class="token keyword">while</span> <span class="token boolean">True</span><span class="token punctuation">:</span>
        page_nums <span class="token operator">=</span> input<span class="token punctuation">(</span><span class="token string">'请输入下载页数（整数，默认为1，每页20）>> '</span><span class="token punctuation">)</span>
        <span class="token keyword">if</span> page_nums<span class="token punctuation">:</span>
            <span class="token keyword">if</span> <span class="token operator">not</span> page_nums<span class="token punctuation">.</span>isdecimal<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
                <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">'您输入的页数有误，必须为整数，请重新输入'</span><span class="token punctuation">)</span>
                <span class="token keyword">continue</span>
            <span class="token keyword">else</span><span class="token punctuation">:</span>
                page_nums <span class="token operator">=</span> int<span class="token punctuation">(</span>page_nums<span class="token punctuation">)</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            page_nums <span class="token operator">=</span> <span class="token number">1</span>
        <span class="token keyword">break</span>

    start<span class="token punctuation">(</span>image_type<span class="token punctuation">,</span> page_nums<span class="token punctuation">)</span>

    print_color<span class="token punctuation">(</span>
        f<span class="token string">'\n\n共检索到 {search_count + downloaded_count} 张图片, 之前已下载 {downloaded_count} 张, 还需下载 {search_count} 张, 本次下载成功 {success_count} 张'</span><span class="token punctuation">)</span>

<span class="token keyword">if</span> __name__ <span class="token operator">==</span> <span class="token string">'__main__'</span><span class="token punctuation">:</span>
    run<span class="token punctuation">(</span><span class="token punctuation">)</span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
</li>
</ul>
<p>运行结果</p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210407120847338.png"></p>
<p>运行演示</p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/spider_netbian.gif"></p>
<h4 id="版本2"><a href="#版本2" class="headerlink" title="版本2"></a>版本2</h4><p>spider.py</p>
<pre class="line-numbers language-python"><code class="language-python"><span class="token keyword">import</span> os
<span class="token keyword">from</span> concurrent<span class="token punctuation">.</span>futures <span class="token keyword">import</span> ThreadPoolExecutor
<span class="token keyword">from</span> urllib<span class="token punctuation">.</span>parse <span class="token keyword">import</span> urljoin

<span class="token keyword">import</span> requests
<span class="token keyword">from</span> lxml <span class="token keyword">import</span> etree
<span class="token keyword">from</span> prettytable <span class="token keyword">import</span> PrettyTable
<span class="token keyword">from</span> tqdm <span class="token keyword">import</span> tqdm
<span class="token keyword">from</span> zyf<span class="token punctuation">.</span>timer <span class="token keyword">import</span> timeit

<span class="token keyword">from</span> utils <span class="token keyword">import</span> print_color

image_dir <span class="token operator">=</span> <span class="token string">'images'</span>

image_type_info <span class="token operator">=</span> <span class="token punctuation">{</span>
    <span class="token number">1</span><span class="token punctuation">:</span> <span class="token string">'4kfengjing'</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">:</span> <span class="token string">'4kmeinv'</span><span class="token punctuation">,</span> <span class="token number">3</span><span class="token punctuation">:</span> <span class="token string">'4kyouxi'</span><span class="token punctuation">,</span> <span class="token number">4</span><span class="token punctuation">:</span> <span class="token string">'4kdongman'</span><span class="token punctuation">,</span>
    <span class="token number">5</span><span class="token punctuation">:</span> <span class="token string">'4kyingshi'</span><span class="token punctuation">,</span> <span class="token number">6</span><span class="token punctuation">:</span> <span class="token string">'4kmingxing'</span><span class="token punctuation">,</span> <span class="token number">7</span><span class="token punctuation">:</span> <span class="token string">'4kqiche'</span><span class="token punctuation">,</span> <span class="token number">8</span><span class="token punctuation">:</span> <span class="token string">'4kdongwu'</span><span class="token punctuation">,</span>
    <span class="token number">9</span><span class="token punctuation">:</span> <span class="token string">'4krenwu'</span><span class="token punctuation">,</span> <span class="token number">10</span><span class="token punctuation">:</span> <span class="token string">'4kmeishi'</span><span class="token punctuation">,</span> <span class="token number">11</span><span class="token punctuation">:</span> <span class="token string">'4kzongjiao'</span><span class="token punctuation">,</span> <span class="token number">12</span><span class="token punctuation">:</span> <span class="token string">'4kbeijing'</span><span class="token punctuation">,</span>
<span class="token punctuation">}</span>
image_tip_info <span class="token operator">=</span> <span class="token punctuation">{</span>
    <span class="token number">1</span><span class="token punctuation">:</span> <span class="token string">'4K风景'</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">:</span> <span class="token string">'4K美女'</span><span class="token punctuation">,</span> <span class="token number">3</span><span class="token punctuation">:</span> <span class="token string">'4K游戏'</span><span class="token punctuation">,</span> <span class="token number">4</span><span class="token punctuation">:</span> <span class="token string">'4K动漫'</span><span class="token punctuation">,</span>
    <span class="token number">5</span><span class="token punctuation">:</span> <span class="token string">'4K影视'</span><span class="token punctuation">,</span> <span class="token number">6</span><span class="token punctuation">:</span> <span class="token string">'4K明星'</span><span class="token punctuation">,</span> <span class="token number">7</span><span class="token punctuation">:</span> <span class="token string">'4K汽车'</span><span class="token punctuation">,</span> <span class="token number">8</span><span class="token punctuation">:</span> <span class="token string">'4K动物'</span><span class="token punctuation">,</span>
    <span class="token number">9</span><span class="token punctuation">:</span> <span class="token string">'4K人物'</span><span class="token punctuation">,</span> <span class="token number">10</span><span class="token punctuation">:</span> <span class="token string">'4K美食'</span><span class="token punctuation">,</span> <span class="token number">11</span><span class="token punctuation">:</span> <span class="token string">'4K宗教'</span><span class="token punctuation">,</span> <span class="token number">12</span><span class="token punctuation">:</span> <span class="token string">'4K背景'</span><span class="token punctuation">,</span>
<span class="token punctuation">}</span>

image_data_list <span class="token operator">=</span> set<span class="token punctuation">(</span><span class="token punctuation">)</span>
search_count <span class="token operator">=</span> <span class="token number">0</span>
downloaded_count <span class="token operator">=</span> <span class="token number">0</span>
success_count <span class="token operator">=</span> <span class="token number">0</span>
history_file_path <span class="token operator">=</span> <span class="token string">'history.txt'</span>

<span class="token keyword">if</span> <span class="token operator">not</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>exists<span class="token punctuation">(</span>image_dir<span class="token punctuation">)</span><span class="token punctuation">:</span>
    os<span class="token punctuation">.</span>makedirs<span class="token punctuation">(</span>image_dir<span class="token punctuation">)</span>

history_file <span class="token operator">=</span> open<span class="token punctuation">(</span>history_file_path<span class="token punctuation">,</span> mode<span class="token operator">=</span><span class="token string">'a+'</span><span class="token punctuation">,</span> encoding<span class="token operator">=</span><span class="token string">'utf-8'</span><span class="token punctuation">)</span>
downloaded_images <span class="token operator">=</span> set<span class="token punctuation">(</span><span class="token punctuation">)</span>
<span class="token keyword">if</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>exists<span class="token punctuation">(</span>history_file_path<span class="token punctuation">)</span><span class="token punctuation">:</span>
    history_file<span class="token punctuation">.</span>seek<span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">)</span>
    downloaded_images <span class="token operator">=</span> <span class="token punctuation">{</span>line<span class="token punctuation">.</span>strip<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">for</span> line <span class="token keyword">in</span> history_file<span class="token punctuation">}</span>


<span class="token keyword">def</span> <span class="token function">init_tips_info</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    table <span class="token operator">=</span> PrettyTable<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token string">'序号1'</span><span class="token punctuation">,</span> <span class="token string">'壁纸分类1'</span><span class="token punctuation">,</span> <span class="token string">'序号2'</span><span class="token punctuation">,</span> <span class="token string">'壁纸分类2'</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
    row <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span>
    <span class="token keyword">for</span> index<span class="token punctuation">,</span> key <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>image_tip_info<span class="token punctuation">,</span> start<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        row<span class="token punctuation">.</span>extend<span class="token punctuation">(</span><span class="token punctuation">[</span>key<span class="token punctuation">,</span> image_tip_info<span class="token punctuation">[</span>key<span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
        <span class="token keyword">if</span> len<span class="token punctuation">(</span>row<span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token number">4</span><span class="token punctuation">:</span>
            table<span class="token punctuation">.</span>add_row<span class="token punctuation">(</span>row<span class="token punctuation">)</span>
            row<span class="token punctuation">.</span>clear<span class="token punctuation">(</span><span class="token punctuation">)</span>
            <span class="token keyword">continue</span>
        <span class="token keyword">if</span> index <span class="token operator">==</span> len<span class="token punctuation">(</span>image_type_info<span class="token punctuation">)</span><span class="token punctuation">:</span>
            row<span class="token punctuation">.</span>extend<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token string">'-'</span><span class="token punctuation">,</span> <span class="token string">'-'</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
            table<span class="token punctuation">.</span>add_row<span class="token punctuation">(</span>row<span class="token punctuation">)</span>
    <span class="token keyword">print</span><span class="token punctuation">(</span>table<span class="token punctuation">)</span>


@timeit
<span class="token keyword">def</span> <span class="token function">start_download</span><span class="token punctuation">(</span>image_type<span class="token punctuation">:</span> int<span class="token punctuation">,</span> page_nums<span class="token punctuation">:</span> int<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">print</span><span class="token punctuation">(</span>f<span class="token string">'正在检索 - {image_tip_info[image_type]} - 请稍等 - 马上开始下载'</span><span class="token punctuation">)</span>

    url <span class="token operator">=</span> <span class="token string">'https://pic.netbian.com'</span>
    <span class="token keyword">if</span> image_type <span class="token operator">!=</span> <span class="token number">0</span><span class="token punctuation">:</span>
        url <span class="token operator">=</span> urljoin<span class="token punctuation">(</span>url<span class="token punctuation">,</span> image_type_info<span class="token punctuation">[</span>image_type<span class="token punctuation">]</span><span class="token punctuation">)</span>
        image_type_desc <span class="token operator">=</span> image_tip_info<span class="token punctuation">[</span>image_type<span class="token punctuation">]</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        url <span class="token operator">=</span> url
        image_type_desc <span class="token operator">=</span> <span class="token string">'最新壁纸'</span>

    <span class="token keyword">if</span> <span class="token operator">not</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>exists<span class="token punctuation">(</span>f<span class="token string">'{image_dir}/{image_type_desc}'</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        os<span class="token punctuation">.</span>mkdir<span class="token punctuation">(</span>f<span class="token string">'{image_dir}/{image_type_desc}'</span><span class="token punctuation">)</span>

    <span class="token keyword">if</span> page_nums <span class="token operator">></span> <span class="token number">1</span><span class="token punctuation">:</span>
        <span class="token keyword">with</span> ThreadPoolExecutor<span class="token punctuation">(</span>max_workers<span class="token operator">=</span>page_nums<span class="token punctuation">)</span> <span class="token keyword">as</span> pool<span class="token punctuation">:</span>
            <span class="token keyword">for</span> page <span class="token keyword">in</span> range<span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> page_nums <span class="token operator">+</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
                pool<span class="token punctuation">.</span>submit<span class="token punctuation">(</span>get_image_data<span class="token punctuation">,</span> url<span class="token punctuation">,</span> image_type_desc<span class="token punctuation">,</span> page<span class="token punctuation">)</span><span class="token punctuation">.</span>add_done_callback<span class="token punctuation">(</span>download_images_callback<span class="token punctuation">)</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        data_list <span class="token operator">=</span> get_image_data<span class="token punctuation">(</span>url<span class="token operator">=</span>url<span class="token punctuation">,</span> image_type_desc<span class="token operator">=</span>image_type_desc<span class="token punctuation">,</span> page_num<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span>
        download_images<span class="token punctuation">(</span>data_list<span class="token punctuation">)</span>


<span class="token keyword">def</span> <span class="token function">get_image_data</span><span class="token punctuation">(</span>url<span class="token punctuation">:</span> str<span class="token punctuation">,</span> image_type_desc<span class="token punctuation">:</span> str<span class="token punctuation">,</span> page_num<span class="token punctuation">:</span> int<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">global</span> downloaded_count<span class="token punctuation">,</span> search_count
    url <span class="token operator">=</span> url <span class="token keyword">if</span> page_num <span class="token operator">==</span> <span class="token number">1</span> <span class="token keyword">else</span> f<span class="token string">'{url}/index_{page_num}.html'</span>
    data_list <span class="token operator">=</span> set<span class="token punctuation">(</span><span class="token punctuation">)</span>
    response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>url<span class="token punctuation">)</span>
    html <span class="token operator">=</span> etree<span class="token punctuation">.</span>HTML<span class="token punctuation">(</span>response<span class="token punctuation">.</span>content<span class="token punctuation">)</span>
    <span class="token keyword">for</span> li <span class="token keyword">in</span> html<span class="token punctuation">.</span>xpath<span class="token punctuation">(</span><span class="token string">'//*[@id="main"]/div[3]/ul/li'</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        href <span class="token operator">=</span> li<span class="token punctuation">.</span>xpath<span class="token punctuation">(</span><span class="token string">'a/@href'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span>
        uid <span class="token operator">=</span> href<span class="token punctuation">.</span>rsplit<span class="token punctuation">(</span><span class="token string">'/'</span><span class="token punctuation">,</span> maxsplit<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'.html'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
        title <span class="token operator">=</span> li<span class="token punctuation">.</span>xpath<span class="token punctuation">(</span><span class="token string">'a//img/@alt'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">' '</span><span class="token punctuation">,</span> <span class="token string">'_'</span><span class="token punctuation">)</span>
        response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>urljoin<span class="token punctuation">(</span>url<span class="token punctuation">,</span> href<span class="token punctuation">)</span><span class="token punctuation">)</span>
        html <span class="token operator">=</span> etree<span class="token punctuation">.</span>HTML<span class="token punctuation">(</span>response<span class="token punctuation">.</span>content<span class="token punctuation">)</span>
        img_url <span class="token operator">=</span> urljoin<span class="token punctuation">(</span>url<span class="token punctuation">,</span> html<span class="token punctuation">.</span>xpath<span class="token punctuation">(</span><span class="token string">'//a[@id="img"]/img/@src'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
        filepath <span class="token operator">=</span> f<span class="token string">'{image_dir}/{image_type_desc}/{title}_{uid}.jpg'</span>
        <span class="token keyword">if</span> filepath <span class="token operator">not</span> <span class="token keyword">in</span> downloaded_images<span class="token punctuation">:</span>
            data_list<span class="token punctuation">.</span>add<span class="token punctuation">(</span><span class="token punctuation">(</span>img_url<span class="token punctuation">,</span> filepath<span class="token punctuation">)</span><span class="token punctuation">)</span>
            search_count <span class="token operator">+=</span> <span class="token number">1</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            downloaded_count <span class="token operator">+=</span> <span class="token number">1</span>
    <span class="token keyword">return</span> data_list


<span class="token keyword">def</span> <span class="token function">download_images_callback</span><span class="token punctuation">(</span>future<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">global</span> success_count
    data_list <span class="token operator">=</span> future<span class="token punctuation">.</span>result<span class="token punctuation">(</span><span class="token punctuation">)</span>
    <span class="token keyword">if</span> len<span class="token punctuation">(</span>data_list<span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token number">0</span><span class="token punctuation">:</span>
        <span class="token keyword">return</span>
    <span class="token keyword">if</span> data_list<span class="token punctuation">:</span>
        task_progress <span class="token operator">=</span> tqdm<span class="token punctuation">(</span>data_list<span class="token punctuation">,</span> ncols<span class="token operator">=</span><span class="token number">100</span><span class="token punctuation">)</span>
    <span class="token keyword">else</span><span class="token punctuation">:</span>
        task_progress <span class="token operator">=</span> tqdm<span class="token punctuation">(</span>image_data_list<span class="token punctuation">,</span> ncols<span class="token operator">=</span><span class="token number">100</span><span class="token punctuation">)</span>
    <span class="token keyword">for</span> index<span class="token punctuation">,</span> task <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>task_progress<span class="token punctuation">,</span> start<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        url<span class="token punctuation">,</span> filepath <span class="token operator">=</span> task
        task_progress<span class="token punctuation">.</span>set_description<span class="token punctuation">(</span>f<span class="token string">"正在下载第 {index} 张图片"</span><span class="token punctuation">)</span>
        response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>url<span class="token operator">=</span>url<span class="token punctuation">)</span>
        <span class="token keyword">with</span> open<span class="token punctuation">(</span>filepath<span class="token punctuation">,</span> mode<span class="token operator">=</span><span class="token string">'wb'</span><span class="token punctuation">)</span> <span class="token keyword">as</span> f<span class="token punctuation">:</span>
            f<span class="token punctuation">.</span>write<span class="token punctuation">(</span>response<span class="token punctuation">.</span>content<span class="token punctuation">)</span>
        history_file<span class="token punctuation">.</span>write<span class="token punctuation">(</span>f<span class="token string">'{filepath}\n'</span><span class="token punctuation">)</span>
        success_count <span class="token operator">+=</span> <span class="token number">1</span>


<span class="token keyword">def</span> <span class="token function">download_images</span><span class="token punctuation">(</span>data_list<span class="token punctuation">:</span> set<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">global</span> success_count
    <span class="token keyword">if</span> <span class="token operator">not</span> data_list<span class="token punctuation">:</span>
        <span class="token keyword">return</span>
    task_progress <span class="token operator">=</span> tqdm<span class="token punctuation">(</span>data_list<span class="token punctuation">,</span> ncols<span class="token operator">=</span><span class="token number">100</span><span class="token punctuation">)</span>
    <span class="token keyword">for</span> index<span class="token punctuation">,</span> task <span class="token keyword">in</span> enumerate<span class="token punctuation">(</span>task_progress<span class="token punctuation">,</span> start<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        url<span class="token punctuation">,</span> filepath <span class="token operator">=</span> task
        task_progress<span class="token punctuation">.</span>set_description<span class="token punctuation">(</span>f<span class="token string">"正在下载第 {index} 张图片"</span><span class="token punctuation">)</span>
        response <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>url<span class="token operator">=</span>url<span class="token punctuation">)</span>
        <span class="token keyword">with</span> open<span class="token punctuation">(</span>filepath<span class="token punctuation">,</span> mode<span class="token operator">=</span><span class="token string">'wb'</span><span class="token punctuation">)</span> <span class="token keyword">as</span> f<span class="token punctuation">:</span>
            f<span class="token punctuation">.</span>write<span class="token punctuation">(</span>response<span class="token punctuation">.</span>content<span class="token punctuation">)</span>
        history_file<span class="token punctuation">.</span>write<span class="token punctuation">(</span>f<span class="token string">'{filepath}\n'</span><span class="token punctuation">)</span>
        success_count <span class="token operator">+=</span> <span class="token number">1</span>


<span class="token keyword">def</span> <span class="token function">run</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
    init_tips_info<span class="token punctuation">(</span><span class="token punctuation">)</span>
    <span class="token keyword">while</span> <span class="token boolean">True</span><span class="token punctuation">:</span>
        image_type <span class="token operator">=</span> input<span class="token punctuation">(</span><span class="token string">'请选择下载的壁纸分类(序号,默认为最新壁纸) >>  '</span><span class="token punctuation">)</span>
        <span class="token keyword">if</span> image_type<span class="token punctuation">:</span>
            <span class="token keyword">if</span> <span class="token operator">not</span> image_type<span class="token punctuation">.</span>isdecimal<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">or</span> int<span class="token punctuation">(</span>image_type<span class="token punctuation">)</span> <span class="token operator">not</span> <span class="token keyword">in</span> image_type_info<span class="token punctuation">:</span>
                <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">'您输入的分类序号有误，请重新确认后输入'</span><span class="token punctuation">)</span>
                <span class="token keyword">continue</span>
            <span class="token keyword">else</span><span class="token punctuation">:</span>
                image_type <span class="token operator">=</span> int<span class="token punctuation">(</span>image_type<span class="token punctuation">)</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            image_type <span class="token operator">=</span> <span class="token number">0</span>
        <span class="token keyword">break</span>
    <span class="token keyword">while</span> <span class="token boolean">True</span><span class="token punctuation">:</span>
        page_nums <span class="token operator">=</span> input<span class="token punctuation">(</span><span class="token string">'请输入下载页数（整数，默认为1，每页20）>> '</span><span class="token punctuation">)</span>
        <span class="token keyword">if</span> page_nums<span class="token punctuation">:</span>
            <span class="token keyword">if</span> <span class="token operator">not</span> page_nums<span class="token punctuation">.</span>isdecimal<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
                <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">'您输入的页数有误，必须为整数，请重新输入'</span><span class="token punctuation">)</span>
                <span class="token keyword">continue</span>
            <span class="token keyword">else</span><span class="token punctuation">:</span>
                page_nums <span class="token operator">=</span> int<span class="token punctuation">(</span>page_nums<span class="token punctuation">)</span>
        <span class="token keyword">else</span><span class="token punctuation">:</span>
            page_nums <span class="token operator">=</span> <span class="token number">1</span>
        <span class="token keyword">break</span>

    start_download<span class="token punctuation">(</span>image_type<span class="token operator">=</span>image_type<span class="token punctuation">,</span> page_nums<span class="token operator">=</span>page_nums<span class="token punctuation">)</span>

    print_color<span class="token punctuation">(</span>
        f<span class="token string">'\n\n共检索到 {search_count + downloaded_count} 张图片, 之前已下载 {downloaded_count} 张, 还需下载 {search_count} 张, 本次下载成功 {success_count} 张'</span><span class="token punctuation">)</span>


<span class="token keyword">if</span> __name__ <span class="token operator">==</span> <span class="token string">'__main__'</span><span class="token punctuation">:</span>
    run<span class="token punctuation">(</span><span class="token punctuation">)</span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre>
<p>运行结果</p>
<p><img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/image/image-20210407120641203.png"></p>
<p>经过测试，版本1和版本2下载同样的100张图片，一个耗时12.686，一个12.961，速度几乎一致。</p>
<p>面向对象代码，有兴趣的伙伴可以进行学习：<a target="_blank" rel="noopener" href="https://gitee.com/zhangyafeii/python-doc-tutorial/tree/master/code/%E7%BD%91%E7%BB%9C%E7%88%AC%E8%99%AB/%E7%88%AC%E8%99%AB%E5%AE%9E%E6%88%98%EF%BC%9A%E9%AB%98%E6%B8%85%E5%9B%BE%E7%89%87%E4%B8%8B%E8%BD%BD">爬虫实战：高清图片下载</a></p>

            </div>
            <hr/>

            

    <div class="reprint" id="reprint-statement">
        
            <div class="reprint__author">
                <span class="reprint-meta" style="font-weight: bold;">
                    <i class="fas fa-user">
                        文章作者:
                    </i>
                </span>
                <span class="reprint-info">
                    <a href="https://zhangyafeii.gitee.io" rel="external nofollow noreferrer">张亚飞</a>
                </span>
            </div>
            <div class="reprint__type">
                <span class="reprint-meta" style="font-weight: bold;">
                    <i class="fas fa-link">
                        文章链接:
                    </i>
                </span>
                <span class="reprint-info">
                    <a href="https://zhangyafeii.gitee.io/python/advanced/pa-chong-shi-zhan-gao-qing-tu-pian-xia-zai/">https://zhangyafeii.gitee.io/python/advanced/pa-chong-shi-zhan-gao-qing-tu-pian-xia-zai/</a>
                </span>
            </div>
            <div class="reprint__notice">
                <span class="reprint-meta" style="font-weight: bold;">
                    <i class="fas fa-copyright">
                        版权声明:
                    </i>
                </span>
                <span class="reprint-info">
                    本博客所有文章除特別声明外，均采用
                    <a href="https://creativecommons.org/licenses/by/4.0/deed.zh" rel="external nofollow noreferrer" target="_blank">CC BY 4.0</a>
                    许可协议。转载请注明来源
                    <a href="https://zhangyafeii.gitee.io" target="_blank">张亚飞</a>
                    !
                </span>
            </div>
        
    </div>

    <script async defer>
      document.addEventListener("copy", function (e) {
        let toastHTML = '<span>复制成功，请遵循本文的转载规则</span><button class="btn-flat toast-action" onclick="navToReprintStatement()" style="font-size: smaller">查看</a>';
        M.toast({html: toastHTML})
      });

      function navToReprintStatement() {
        $("html, body").animate({scrollTop: $("#reprint-statement").offset().top - 80}, 800);
      }
    </script>



            <div class="tag_share" style="display: block;">
                <div class="post-meta__tag-list" style="display: inline-block;">
                    
                        <div class="article-tag">
                            
                                <a href="/tags/Python%E4%B9%8B%E8%B7%AF/">
                                    <span class="chip bg-color">Python之路</span>
                                </a>
                            
                                <a href="/tags/%E8%BF%9B%E9%98%B6%E7%AF%87/">
                                    <span class="chip bg-color">进阶篇</span>
                                </a>
                            
                                <a href="/tags/%E7%BD%91%E7%BB%9C%E7%88%AC%E8%99%AB/">
                                    <span class="chip bg-color">网络爬虫</span>
                                </a>
                            
                                <a href="/tags/%E9%AB%98%E6%B8%85%E5%9B%BE%E7%89%87%E4%B8%8B%E8%BD%BD/">
                                    <span class="chip bg-color">高清图片下载</span>
                                </a>
                            
                        </div>
                    
                </div>
                <div class="post_share" style="zoom: 80%; width: fit-content; display: inline-block; float: right; margin: -0.15rem 0;">
                    <link rel="stylesheet" type="text/css" href="/libs/share/css/share.min.css">

<div id="article-share">
    
    
    <div class="social-share" data-sites="twitter,facebook,google,qq,qzone,wechat,weibo,douban,linkedin" data-wechat-qrcode-helper="<p>微信扫一扫即可分享！</p>"></div>
    <script src="/libs/share/js/social-share.min.js"></script>
    

    

</div>

                </div>
            </div>
            
                <style>
    #reward {
        margin: 40px 0;
        text-align: center;
    }

    #reward .reward-link {
        font-size: 1.4rem;
        line-height: 38px;
    }

    #reward .btn-floating:hover {
        box-shadow: 0 6px 12px rgba(0, 0, 0, 0.2), 0 5px 15px rgba(0, 0, 0, 0.2);
    }

    #rewardModal {
        width: 320px;
        height: 350px;
    }

    #rewardModal .reward-title {
        margin: 15px auto;
        padding-bottom: 5px;
    }

    #rewardModal .modal-content {
        padding: 10px;
    }

    #rewardModal .close {
        position: absolute;
        right: 15px;
        top: 15px;
        color: rgba(0, 0, 0, 0.5);
        font-size: 1.3rem;
        line-height: 20px;
        cursor: pointer;
    }

    #rewardModal .close:hover {
        color: #ef5350;
        transform: scale(1.3);
        -moz-transform:scale(1.3);
        -webkit-transform:scale(1.3);
        -o-transform:scale(1.3);
    }

    #rewardModal .reward-tabs {
        margin: 0 auto;
        width: 210px;
    }

    .reward-tabs .tabs {
        height: 38px;
        margin: 10px auto;
        padding-left: 0;
    }

    .reward-content ul {
        padding-left: 0 !important;
    }

    .reward-tabs .tabs .tab {
        height: 38px;
        line-height: 38px;
    }

    .reward-tabs .tab a {
        color: #fff;
        background-color: #ccc;
    }

    .reward-tabs .tab a:hover {
        background-color: #ccc;
        color: #fff;
    }

    .reward-tabs .wechat-tab .active {
        color: #fff !important;
        background-color: #22AB38 !important;
    }

    .reward-tabs .alipay-tab .active {
        color: #fff !important;
        background-color: #019FE8 !important;
    }

    .reward-tabs .reward-img {
        width: 210px;
        height: 210px;
    }
</style>

<div id="reward">
    <a href="#rewardModal" class="reward-link modal-trigger btn-floating btn-medium waves-effect waves-light red">赏</a>

    <!-- Modal Structure -->
    <div id="rewardModal" class="modal">
        <div class="modal-content">
            <a class="close modal-close"><i class="fas fa-times"></i></a>
            <h4 class="reward-title">你的赏识是我前进的动力</h4>
            <div class="reward-content">
                <div class="reward-tabs">
                    <ul class="tabs row">
                        <li class="tab col s6 alipay-tab waves-effect waves-light"><a href="#alipay">支付宝</a></li>
                        <li class="tab col s6 wechat-tab waves-effect waves-light"><a href="#wechat">微 信</a></li>
                    </ul>
                    <div id="alipay">
                        <img src="/medias/reward/alipay.jpg" class="reward-img" alt="支付宝打赏二维码">
                    </div>
                    <div id="wechat">
                        <img src="/medias/reward/wechat.png" class="reward-img" alt="微信打赏二维码">
                    </div>
                </div>
            </div>
        </div>
    </div>
</div>

<script>
    $(function () {
        $('.tabs').tabs();
    });
</script>
            
        </div>
    </div>

    
        <link rel="stylesheet" href="/libs/gitalk/gitalk.css">
<link rel="stylesheet" href="/css/my-gitalk.css">

<div class="card gitalk-card" data-aos="fade-up">
    <div class="comment_headling" style="font-size: 20px; font-weight: 700; position: relative; left: 20px; top: 15px; padding-bottom: 5px;">
        <i class="fas fa-comments fa-fw" aria-hidden="true"></i>
        <span>评论</span>
    </div>
    <div id="gitalk-container" class="card-content"></div>
</div>

<script src="/libs/gitalk/gitalk.min.js"></script>
<script>
    let gitalk = new Gitalk({
        clientID: 'b9bf0e29501275f4f23c',
        clientSecret: '1c56bc88fdb7b23ee3712916a38e5e8d1311823a',
        repo: 'github.io',
        owner: 'zhangyafeii',
        admin: "zhangyafeii",
        id: '2021-04-06T20-00-01',
        distractionFreeMode: false  // Facebook-like distraction free mode
    });

    gitalk.render('gitalk-container');
</script>
    

    

    

    

    

    

<article id="prenext-posts" class="prev-next articles">
    <div class="row article-row">
        
        <div class="article col s12 m6" data-aos="fade-up" data-aos="fade-up">
            <div class="article-badge left-badge text-color">
                <i class="far fa-dot-circle"></i>&nbsp;本篇
            </div>
            <div class="card">
                <a href="/python/advanced/pa-chong-shi-zhan-gao-qing-tu-pian-xia-zai/">
                    <div class="card-image">
                        
                        <img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/Python/blog/advanced-6.jpg" class="responsive-img" alt="爬虫实战：高清图片下载">
                        
                        <span class="card-title">爬虫实战：高清图片下载</span>
                    </div>
                </a>
                <div class="card-content article-content">
                    <div class="summary block-with-text">
                        
                            并发编程是编程语言学习中非常重要的知识，但是这块知识中一些概念确实相当难理解，一些概念区分起来很难，比如并发和并行，同步和异步，阻塞和非阻塞，但是这些概念却很重要。因此在此把它总结下来。
                        
                    </div>
                    <div class="publish-info">
                            <span class="publish-date">
                                <i class="far fa-clock fa-fw icon-date"></i>2021-04-06
                            </span>
                        <span class="publish-author">
                            
                            <i class="fas fa-bookmark fa-fw icon-category"></i>
                            
                            <a href="/categories/Python/" class="post-category">
                                    Python
                                </a>
                            
                            <a href="/categories/Python/Advanced/" class="post-category">
                                    Advanced
                                </a>
                            
                            
                        </span>
                    </div>
                </div>

                
                <div class="card-action article-tags">
                    
                    <a href="/tags/Python%E4%B9%8B%E8%B7%AF/">
                        <span class="chip bg-color">Python之路</span>
                    </a>
                    
                    <a href="/tags/%E8%BF%9B%E9%98%B6%E7%AF%87/">
                        <span class="chip bg-color">进阶篇</span>
                    </a>
                    
                    <a href="/tags/%E7%BD%91%E7%BB%9C%E7%88%AC%E8%99%AB/">
                        <span class="chip bg-color">网络爬虫</span>
                    </a>
                    
                    <a href="/tags/%E9%AB%98%E6%B8%85%E5%9B%BE%E7%89%87%E4%B8%8B%E8%BD%BD/">
                        <span class="chip bg-color">高清图片下载</span>
                    </a>
                    
                </div>
                
            </div>
        </div>
        
        
        <div class="article col s12 m6" data-aos="fade-up">
            <div class="article-badge right-badge text-color">
                下一篇&nbsp;<i class="fas fa-chevron-right"></i>
            </div>
            <div class="card">
                <a href="/python/base/di-31-pian-asyncio-yi-bu-bian-cheng/">
                    <div class="card-image">
                        
                        <img src="https://zhangyafei-1258643511.cos.ap-nanjing.myqcloud.com/Python%E6%96%87%E6%A1%A3/assets/31.jpg" class="responsive-img" alt="第31篇：asyncio异步编程">
                        
                        <span class="card-title">第31篇：asyncio异步编程</span>
                    </div>
                </a>
                <div class="card-content article-content">
                    <div class="summary block-with-text">
                        
                            1. 协程协程不是计算机提供，程序员人为创造
协程（Coroutine），也可以被称为微线程，是一种用户态内的上下文切换技术，其实就是一个线程实现代码块相互切换执行。例如：
def func1():
    print(1)
    ...
                        
                    </div>
                    <div class="publish-info">
                            <span class="publish-date">
                                <i class="far fa-clock fa-fw icon-date"></i>2021-04-01
                            </span>
                        <span class="publish-author">
                            
                            <i class="fas fa-bookmark fa-fw icon-category"></i>
                            
                            <a href="/categories/Python/" class="post-category">
                                    Python
                                </a>
                            
                            <a href="/categories/Python/base/" class="post-category">
                                    base
                                </a>
                            
                            
                        </span>
                    </div>
                </div>
                
                <div class="card-action article-tags">
                    
                    <a href="/tags/Python%E4%B9%8B%E8%B7%AF/">
                        <span class="chip bg-color">Python之路</span>
                    </a>
                    
                    <a href="/tags/%E5%BC%82%E6%AD%A5%E7%BC%96%E7%A8%8B/">
                        <span class="chip bg-color">异步编程</span>
                    </a>
                    
                </div>
                
            </div>
        </div>
        
    </div>
</article>

</div>


<script>
    $('#articleContent').on('copy', function (e) {
        // IE8 or earlier browser is 'undefined'
        if (typeof window.getSelection === 'undefined') return;

        var selection = window.getSelection();
        // if the selection is short let's not annoy our users.
        if (('' + selection).length < Number.parseInt('120')) {
            return;
        }

        // create a div outside of the visible area and fill it with the selected text.
        var bodyElement = document.getElementsByTagName('body')[0];
        var newdiv = document.createElement('div');
        newdiv.style.position = 'absolute';
        newdiv.style.left = '-99999px';
        bodyElement.appendChild(newdiv);
        newdiv.appendChild(selection.getRangeAt(0).cloneContents());

        // we need a <pre> tag workaround.
        // otherwise the text inside "pre" loses all the line breaks!
        if (selection.getRangeAt(0).commonAncestorContainer.nodeName === 'PRE') {
            newdiv.innerHTML = "<pre>" + newdiv.innerHTML + "</pre>";
        }

        var url = document.location.href;
        newdiv.innerHTML += '<br />'
            + '来源: 飞凡空间<br />'
            + '文章作者: 张亚飞<br />'
            + '文章链接: <a href="' + url + '">' + url + '</a><br />'
            + '本文章著作权归作者所有，任何形式的转载都请注明出处。';

        selection.selectAllChildren(newdiv);
        window.setTimeout(function () {bodyElement.removeChild(newdiv);}, 200);
    });
</script>


<!-- 代码块功能依赖 -->
<script type="text/javascript" src="/libs/codeBlock/codeBlockFuction.js"></script>

<!-- 代码语言 -->

<script type="text/javascript" src="/libs/codeBlock/codeLang.js"></script>

    
<!-- 代码块复制 -->

<script type="text/javascript" src="/libs/codeBlock/codeCopy.js"></script>


<!-- 代码块收缩 -->

<script type="text/javascript" src="/libs/codeBlock/codeShrink.js"></script>


<!-- 代码块折行 -->

<style type="text/css">
code[class*="language-"], pre[class*="language-"] { white-space: pre !important; }
</style>

    </div>
    <div id="toc-aside" class="expanded col l3 hide-on-med-and-down">
        <div class="toc-widget">
            <div class="toc-title"><i class="far fa-list-alt"></i>&nbsp;&nbsp;目录</div>
            <div id="toc-content"></div>
        </div>
    </div>
</div>

<!-- TOC 悬浮按钮. -->

<div id="floating-toc-btn" class="hide-on-med-and-down">
    <a class="btn-floating btn-large bg-color">
        <i class="fas fa-list-ul"></i>
    </a>
</div>


<script src="/libs/tocbot/tocbot.min.js"></script>
<script>
    $(function () {
        tocbot.init({
            tocSelector: '#toc-content',
            contentSelector: '#articleContent',
            headingsOffset: -($(window).height() * 0.4 - 45),
            // headingsOffset: -205,
            headingSelector: 'h2, h3, h4'
        });

        // modify the toc link href to support Chinese.
        let i = 0;
        let tocHeading = 'toc-heading-';
        $('#toc-content a').each(function () {
            $(this).attr('href', '#' + tocHeading + (++i));
        });

        // modify the heading title id to support Chinese.
        i = 0;
        $('#articleContent').children('h2, h3, h4').each(function () {
            $(this).attr('id', tocHeading + (++i));
        });

        // Set scroll toc fixed.
        let tocHeight = parseInt($(window).height() * 0.4 - 64);
        let $tocWidget = $('.toc-widget');
        $(window).scroll(function () {
            let scroll = $(window).scrollTop();
            /* add post toc fixed. */
            if (scroll > tocHeight) {
                $tocWidget.addClass('toc-fixed');
            } else {
                $tocWidget.removeClass('toc-fixed');
            }
        });

        
        /* 修复文章卡片 div 的宽度. */
        let fixPostCardWidth = function (srcId, targetId) {
            let srcDiv = $('#' + srcId);
            if (srcDiv.length === 0) {
                return;
            }

            let w = srcDiv.width();
            if (w >= 450) {
                w = w + 21;
            } else if (w >= 350 && w < 450) {
                w = w + 18;
            } else if (w >= 300 && w < 350) {
                w = w + 16;
            } else {
                w = w + 14;
            }
            $('#' + targetId).width(w);
        };

        // 切换TOC目录展开收缩的相关操作.
        const expandedClass = 'expanded';
        let $tocAside = $('#toc-aside');
        let $mainContent = $('#main-content');
        $('#floating-toc-btn .btn-floating').click(function () {
            if ($tocAside.hasClass(expandedClass)) {
                $tocAside.removeClass(expandedClass).hide();
                $mainContent.removeClass('l9');
            } else {
                $tocAside.addClass(expandedClass).show();
                $mainContent.addClass('l9');
            }
            fixPostCardWidth('artDetail', 'prenext-posts');
        });
        
    });
</script>

    

</main>



    <footer class="page-footer bg-color">
    <div class="container row center-align">
        <div class="col s12 m8 l8 copy-right">
            Copyright&nbsp;&copy;
            <span id="year">年份</span>
            <a href="https://zhangyafeii.gitee.io" target="_blank">张亚飞</a>
            |&nbsp;Powered by&nbsp;<a href="https://hexo.io/" target="_blank">Hexo</a>
            |&nbsp;Theme&nbsp;<a href="https://github.com/blinkfox/hexo-theme-matery" target="_blank">Matery</a>
            <br>
            
            &nbsp;<i class="fas fa-chart-area"></i>&nbsp;站点总字数:&nbsp;<span
                class="white-color">344.5k</span>&nbsp;字
            
            
            
            
            
            
            <span id="busuanzi_container_site_pv">
                |&nbsp;<i class="far fa-eye"></i>&nbsp;总访问量:&nbsp;<span id="busuanzi_value_site_pv"
                    class="white-color"></span>&nbsp;次
            </span>
            
            
            <span id="busuanzi_container_site_uv">
                |&nbsp;<i class="fas fa-users"></i>&nbsp;总访问人数:&nbsp;<span id="busuanzi_value_site_uv"
                    class="white-color"></span>&nbsp;人
            </span>
            
            <br>
            
            <span id="sitetime">载入运行时间...</span>
            <script>
                function siteTime() {
                    window.setTimeout("siteTime()", 1000);
                    var seconds = 1000;
                    var minutes = seconds * 60;
                    var hours = minutes * 60;
                    var days = hours * 24;
                    var years = days * 365;
                    var today = new Date();
                    var startYear = "2020";
                    var startMonth = "12";
                    var startDate = "18";
                    var startHour = "12";
                    var startMinute = "12";
                    var startSecond = "12";
                    var todayYear = today.getFullYear();
                    var todayMonth = today.getMonth() + 1;
                    var todayDate = today.getDate();
                    var todayHour = today.getHours();
                    var todayMinute = today.getMinutes();
                    var todaySecond = today.getSeconds();
                    var t1 = Date.UTC(startYear, startMonth, startDate, startHour, startMinute, startSecond);
                    var t2 = Date.UTC(todayYear, todayMonth, todayDate, todayHour, todayMinute, todaySecond);
                    var diff = t2 - t1;
                    var diffYears = Math.floor(diff / years);
                    var diffDays = Math.floor((diff / days) - diffYears * 365);
                    var diffHours = Math.floor((diff - (diffYears * 365 + diffDays) * days) / hours);
                    var diffMinutes = Math.floor((diff - (diffYears * 365 + diffDays) * days - diffHours * hours) /
                        minutes);
                    var diffSeconds = Math.floor((diff - (diffYears * 365 + diffDays) * days - diffHours * hours -
                        diffMinutes * minutes) / seconds);
                    if (startYear == todayYear) {
                        document.getElementById("year").innerHTML = todayYear;
                        document.getElementById("sitetime").innerHTML = "本站已安全运行 " + diffDays + " 天 " + diffHours +
                            " 小时 " + diffMinutes + " 分钟 " + diffSeconds + " 秒";
                    } else {
                        document.getElementById("year").innerHTML = startYear + " - " + todayYear;
                        document.getElementById("sitetime").innerHTML = "本站已安全运行 " + diffYears + " 年 " + diffDays +
                            " 天 " + diffHours + " 小时 " + diffMinutes + " 分钟 " + diffSeconds + " 秒";
                    }
                }
                setInterval(siteTime, 1000);
            </script>
            
            <br>
            
        </div>
        <div class="col s12 m4 l4 social-link social-statis">
    <a href="https://github.com/zhangyafeii" class="tooltipped" target="_blank" data-tooltip="访问我的GitHub" data-position="top" data-delay="50">
        <i class="fab fa-github"></i>
    </a>



    <a href="https://gitee.com/zhangyafeii" class="tooltipped" target="_blank" data-tooltip="访问我的Gitee" data-position="top" data-delay="50">
        <i class="fab fa-google-plus"></i>
    </a>



    <a href="mailto:zhangyafeii@foxmail.com" class="tooltipped" target="_blank" data-tooltip="邮件联系我" data-position="top" data-delay="50">
        <i class="fas fa-envelope-open"></i>
    </a>







    <a href="tencent://AddContact/?fromId=50&fromSubId=1&subcmd=all&uin=1271570224" class="tooltipped" target="_blank" data-tooltip="QQ联系我: 1271570224" data-position="top" data-delay="50">
        <i class="fab fa-qq"></i>
    </a>







    <a href="/atom.xml" class="tooltipped" target="_blank" data-tooltip="RSS 订阅" data-position="top" data-delay="50">
        <i class="fas fa-rss"></i>
    </a>

</div>
    </div>
</footer>

<div class="progress-bar"></div>


    <!-- 搜索遮罩框 -->
<div id="searchModal" class="modal">
    <div class="modal-content">
        <div class="search-header">
            <span class="title"><i class="fas fa-search"></i>&nbsp;&nbsp;搜索</span>
            <input type="search" id="searchInput" name="s" placeholder="请输入搜索的关键字"
                   class="search-input">
        </div>
        <div id="searchResult"></div>
    </div>
</div>

<script src="/js/search.js"></script>
<script type="text/javascript">
$(function () {
    searchFunc("/" + "search.xml", 'searchInput', 'searchResult');
});
</script>
    <!-- 回到顶部按钮 -->
<div id="backTop" class="top-scroll">
    <a class="btn-floating btn-large waves-effect waves-light" href="#!">
        <i class="fas fa-arrow-up"></i>
    </a>
</div>


    <script src="/libs/materialize/materialize.min.js"></script>
    <script src="/libs/masonry/masonry.pkgd.min.js"></script>
    <script src="/libs/aos/aos.js"></script>
    <script src="/libs/scrollprogress/scrollProgress.min.js"></script>
    <script src="/libs/lightGallery/js/lightgallery-all.min.js"></script>
    <script src="/js/matery.js"></script>
    <script src="/js/cursor.js"></script>

    <!-- Global site tag (gtag.js) - Google Analytics -->


    <!-- Baidu Analytics -->

    <!-- Baidu Push -->

<script>
    (function () {
        var bp = document.createElement('script');
        var curProtocol = window.location.protocol.split(':')[0];
        if (curProtocol === 'https') {
            bp.src = 'https://zz.bdstatic.com/linksubmit/push.js';
        } else {
            bp.src = 'http://push.zhanzhang.baidu.com/push.js';
        }
        var s = document.getElementsByTagName("script")[0];
        s.parentNode.insertBefore(bp, s);
    })();
</script>

    
    <script src="/libs/others/clicklove.js" async="async"></script>
    
    
    <script async src="/libs/others/busuanzi.pure.mini.js"></script>
    

    

    

    

    

    
    <script type="text/javascript" src="/libs/background/ribbon-dynamic.js" async="async"></script>
    
    
    
    <script src="/libs/instantpage/instantpage.js" type="module"></script>
    

</body>

</html>
